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Abstract 

The  use  of  Systems  Engineering  (SE)  is  mandated  by  the  Department  of  Defense 
(DoD)  and  Einited  States  Air  Eoree  (EiSAE)  poliey  and  is  to  be  eonsidered  under  the 
purview  of  the  Program  Manager  (PM).  A  normal  SE  program  ean  eonsist  of  multiple 
proeesses  from  user  requirement  generation  to  the  verifieation  and  validation  of  the 
system  under  design.  The  SE  proeess  eneompasses  the  entire  aequisition  program  and 
ean  take  multiple  years  to  eonduct  with  eompletion  only  being  aehieved  when  the 
program  is  disposed  of  at  the  end  of  its  life. 

Rapid  aequisition  programs  sueh  as  those  fulfilling  a  Joint  EJrgent  Operational 
Need  (JEJON)  ean  have  timelines  that  are  eompressed  to  less  than  24  months  from  the 
moment  the  capability  gap  is  recognized  to  the  time  that  the  system  is  put  into  operational 
use.  This  compressed  timeline  often  necessitates  the  truncation  of  some  tasks  and  the 
removal  of  others. 

This  research  examines  the  literature  on  how  the  EJSAE  completes  rapid 
acquisitions  and  compares  it  to  the  responses  of  twelve  members  of  the  acquisition 
community  with  experience  in  rapid  acquisition.  The  data  is  categorized  to  allow  for  the 
main  points  to  be  collected  explaining  how  the  EJSAE  tailors  the  acquisition  and  SE 
processes.  The  results  showed  that  while  some  programs  do  follow  prescribed 
instructions,  most  use  an  ad-hoc  execution  process,  and  the  Systems  Engineering 
Technical  Management  Processes  were  underutilized. 
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TAILORING  SYSTEMS  ENGINEERING  FOR  RAPID  ACQUISITION 


I.  Introduction 

General  Issue 

The  use  of  Systems  Engineering  (SE)  in  aequisition  programs  is  mandated  by 
Department  of  Defense  (DoD)  and  United  States  Air  Eoree  (USAE)  poliey  and  is  to  be 
eonsidered  under  the  purview  of  the  Program  Manager  (PM).  A  typieal  SE  aequisition 
program  ean  eonsist  of  multiple  proeesses  from  user  requirements  generation  to  the 
verifieation  and  validation  of  the  system  under  design.  The  SE  process  parallels  the 
entire  acquisition  program  and  typically  takes  multiple  years,  or  even  decades,  to 
complete. 

Rapid  acquisition  programs,  such  as  those  fulfilling  an  Urgent  Operational  Need 
(UON)  or  JUON,  can  have  timelines  that  are  compressed  to  less  than  24  months  from  the 
moment  the  capability  gap  is  recognized  to  the  time  the  system  is  put  into  operational 
use.  This  compressed  timeline  necessitates  the  truncation  of  some  tasks  and  the 
elimination  of  others.  This  research  examines  the  SE  and  acquisition  processes  that  are 
implemented  by  different  members  of  the  acquisition  community  to  understand  how  they 
tailor  the  processes  to  meet  expedited  timelines  associated  with  rapid  acquisition 
programs. 

Currently,  the  Chief  Systems  Engineer  and  PM  decide  what  system  engineering 
activities  will  be  completed  in  accordance  with  DoD  and  USAE  policy.  This  means  the 
experience  level  of  both  the  Chief  Systems  Engineer  and  the  PM  will  heavily  influence 
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what  they  perceive  as  value  added  products  and  required  documentation.  As  SE  is  very 
broad  and  rapid  acquisition  programs  are  constrained  by  the  expedited  approach,  the 
program  will  not  have  enough  time  to  allow  for  all  systems  engineering  activities  to  be 
completed. 

Problem  Statement 

With  standard  acquisition  practices  taking  too  long  to  be  responsive  to  the  urgent 
needs  of  a  warfighter  currently  engaged  in  operations  around  the  world,  how  does  the 
acquisition  community  in  the  Air  Force  tailor  their  process  to  meet  that  user’s  needs? 

This  research  investigates  the  different  acquisition  and  SE  processes  used  in  rapid 
acquisition  programs  and  compares  them  to  the  military  instructions.  The  objective  of 
the  research  is  to  better  understand  the  different  ways  programs  are  managed  and  how  the 
SE  processes  are  used  during  the  lifecycle  of  these  programs. 

Rapid  Acquisition 

The  DoD  categorizes  its  acquisition  programs  based  upon  the  amount  of  money 
allocated  to  different  parts  of  the  program.  Acquisition  Program  Category  (ACAT)  I 
include  programs  over  $1  Billion  in  research  and  development  funds  (Office  of  the 
Undersecretary  of  Defense  for  Acquisition  Technology  &  Eogistics,  2008).  These  are  the 
major  programs  of  the  DoD  that  take  years  to  develop;  however,  not  all  programs  reach 
this  level  of  cost  or  schedules.  Rapid  Acquisition  programs  are  considered  streamlined 
programs  that  “rapidly  produce  and  deliver  capabilities”  (Joint  Chiefs  of  Staff,  2012). 
Many  programs  are  considered  rapid  acquisitions,  in  which  the  entire  program  only  has 
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eighteen  to  twenty-four  months  between  when  the  requirements  are  initiated  and  when 
the  program  is  fielded. 

New  policy  being  published  by  the  DoD  will  classify  its  acquisition  programs  by 
schedule,  along  with  the  cost  associated  to  the  program.  This  means  there  are  now  three 
new  stratifications  for  projects:  1)  rapid,  which  consist  of  programs  that  are  scheduled  for 
less  than  two  years  of  acquisition  time  before  fielding;  2)  emergent,  which  consists  of 
programs  that  are  scheduled  for  two  to  six  years  of  acquisition  time  before  fielding;  and 
3)  legacy,  which  is  all  programs  that  will  take  more  than  six  years  of  acquisition  time  to 
go  from  need  validation  to  initial  fielding  (Office  Of  The  Undersecretary  Of  Defense  For 
Acquisition  Technology  &  Logistics,  2013). 

The  DoD  considers  JUONS  as  rapid  acquisition  and  removes  them  from  the 
standard  acquisition  strategy  (Gansler  &  Hughes,  2009).  All  DoD  acquisition  programs 
are  required  by  federal  regulations  to  include  systems  engineering  in  their  processes. 
However,  inside  of  a  compressed  time  schedule  there  is  limited  time  available  for  most 
SE  processes.  As  will  be  seen  in  the  literature  review,  there  is  no  guidance  as  to  which 
activities  will  provide  the  most  benefit  for  the  time  invested. 

Methodology 

This  research  was  designed  to  answer  how  the  USAF  tailors  systems  engineering 
and  acquisition  programs  to  complete  rapid  acquisitions.  The  researcher  conducted 
interviews  with  twelve  members  of  the  Air  Force’s  rapid  acquisition  community 
spanning  three  laboratories,  two  traditional  system  program  offices  (SPO),  and  two  rapid 
development  system  program  offices  inside  the  Air  Force  Fife  Cycle  Management  Center 
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(AFLCMC).  By  using  a  broad  population  sample  from  across  the  Air  Force  the  data  was 
able  to  be  triangulated  to  improve  the  internal  validity  of  the  research. 

The  subject  matter  experts  (SMEs)  interviewed  were  selected  for  their  experience 
in  rapid  acquisition  and  by  their  availability  to  the  researcher.  Twelve  SMEs  were 
interviewed  from  a  variety  of  organizations;  however,  due  to  limitations  of  time,  money 
and  access,  not  all  DoD  organizations  that  conduct  rapid  acquisition  were  included  in  this 
study. 

The  data  collected  from  the  interviews  was  coded  and  categorized  based  upon  the 
content  and  used  to  answer  the  basic  questions  posed  in  this  thesis;  i.e.  how  does  the 
USAE  conduct  rapid  acquisition? 

Investigative  Questions 

With  the  inconsistent  implementation  of  tailored  acquisition  and  SE  in  mind,  this 
thesis  focuses  on  understanding  which  acquisition  and  SE  activities  should  be  conducted 
to  help  the  acquisition  programs  in  meeting  the  user’s  needs.  The  following  five 
questions  were  investigated  during  this  thesis. 

1 .  What  processes  does  the  Elnited  States  Air  Eorce  use  to  complete  rapid  acquisition 
projects  and  programs? 

2.  Are  the  observed  processes  consistent  with  prescribed  instructions? 

3.  What  SE  activities  are  used  in  rapid  acquisition  programs  in  the  Elnited  States  Air 
Eorce? 

4.  How  are  rapid  acquisition  programs  tailored  in  the  United  States  Air  Eorce? 

5.  Which  program  attributes  are  used  to  determine  program  tailoring? 
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Summary 

This  chapter  introduced  the  issues  that  are  facing  rapid  acquisition  in  the  DoD  and 
Air  Force.  There  have  been  multiple  attempts  to  accelerate  the  traditional  rapid 
acquisition  process  to  allow  for  faster  responses  to  the  warfighter.  This  thesis  examines 
how  the  acquisition  professionals  in  the  Air  Force  conduct  rapid  acquisition  and  the 
Systems  Engineering  activities  required  to  meet  the  expedited  timelines.  Chapter  2  will 
discuss  the  prescribed  processes  defined  by  the  organizations  that  conduct  rapid 
acquisition  along  with  the  literature  review  of  previous  inquiries  analogous  to  this  study. 
Chapter  3  will  provide  the  methodology  of  the  study.  Chapter  4  will  present  the  results 
of  the  interviews  conducted,  and  Chapter  5  will  examine  the  results  and  give 
recommendations  for  future  research  and  improvements  for  the  study. 
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II.  Literature  Review 


Chapter  Overview 

The  purpose  of  this  ehapter  is  to  review  the  published  literature  in  the  domain  of 
systems  engineering  along  with  doeumentation  deseribing  what  is  required  to  be 
eompleted  in  the  subset  of  rapid  aequisition.  This  overview  lays  the  groundwork  for  the 
researeh  questions  outlined  in  the  previous  ehapter. 

Description 

The  DoD  is  mandated  to  use  three  proeesses  to  develop  new  systems  and 
eapabilities;  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS),  the 
Planning,  Programming,  Budgeting  and  Execution  System  (PPBE)  and  the  Defense 
Acquisition  System  (DAS)  (Sullivan,  2009).  Typical  acquisition  programs  take 
anywhere  from  5-15  years,  with  some  major  programs  such  as  aircraft  or  naval  vessels 
taking  even  longer  (Sullivan,  2009).  Examples  include  the  E-22  Air  Superiority  Eighter 
which  entered  Demonstration  and  Validation  Phase  in  1986  and  didn’t  reach  its  initial 
operation  capability  until  2005,  and  the  Navy’s  newest  nuclear  aircraft  carrier,  the  USS 
Gerald  R.  Eord  which  the  Navy  began  funding  and  development  in  PY2001  and  won’t  be 
delivered  to  the  Navy  until  2016  (Department  of  the  Air  Eorce,  2008)  (Department  of  the 
Navy,  2005;  Department  of  the  Navy,  2013). 

Legacy  Acquisition 

DoD  5000.1  and  5000.2  are  the  formal  instructions  defining  the  way  the  military 
acquires  new  weapon  systems  and  capabilities.  Eirst  published  in  1971  and  evolving  out 
of  the  Cold-War  policies  and  dictated  by  federal  statutes,  the  avenues  for  acquiring 
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weapons  systems  and  capabilities  are  organized  into  a  series  of  decision  gates  allowing  a 
program  to  progress  from  one  phase  to  the  next  contingent  on  demonstrating  progress 
towards  program  objectives  and  user  requirements  (Ferrara,  1996).  As  stated  in  the 
current  version  of  the  instruction,  “evolutionary  acquisition  is  the  preferred  DoD  strategy 
for  rapid  acquisition  of  mature  technology  for  the  user”  (Office  of  the  Undersecretary  of 
Defense  for  Acquisition  Technology  &  Logistics,  2008).  However,  as  discussed  above 


and  seen  below  in  Figure  1 ,  this  is  not  always  the  case. 


Figure  1  Program  Lifecycle  (Defense  Acquisition  University,  2014) 


Starting  from  the  left  of  Figure  1,  a  requirement  is  validated  and  then  the  program 
moves  from  the  left  to  right,  going  from  the  Material  Solution  Analysis  Phase  to  the 
Technology  Development  Phase,  later  to  the  Engineering  &  Manufacturing  Development 
Phase,  then  to  the  Production  &  Development  Phase  and  finally  onto  Operations  and 
Sustainment.  Based  upon  the  expected  cost  of  the  programs,  they  will  be  categorized  as 
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an  AC  AT  Level  I,  II,  or  III  and,  as  sueh,  the  AC  AT  level  I  and  II  programs  will  be 
required  to  eomplete  more  of  the  aetivities  shown  in  the  chart  than  those  programs 
designated  as  ACAT  level  III  (Office  of  the  Undersecretary  of  Defense  for  Acquisition 
Technology  &  Logistics,  2008). 

Need  for  Rapid  Acquisition 

The  longer  timelines  of  legacy  acquisitions  are  one  of  the  reasons  why  the 
military  uses  JUONs  to  establish  rapid  acquisition  programs  that  will  meet  an  operational 
need  within  18  to  24  months.  Examples  of  these  accelerated  programs  include  the  Mine 
Resistant  Ambush  Protection  (MRAP)  Vehicle  which  was  initiated  in  2007  and  delivered 
vehicles  by  2008,  and  the  Project  Liberty  aircraft  in  which,  inside  a  year  of  receiving  the 
warfighters  need  statement,  the  United  States  Air  Force  received  their  first  airframe  for 
deployment  (Force,  2010)  (Sullivan,  2009). 

To  meet  the  timelines  associated  with  rapid  acquisition,  certain  processes 
normally  required  under  the  JCIDS,  PPBE  and  DAS  are  shortened  while  others  are 
eliminated  or  completed  after  the  initial  fielding  of  the  system.  Per  military  instruction 
rapid  acquisition  is: 

A  streamlined  and  tightly  integrated  iterative  approach,  acting  upon  validated 
urgent  or  emergent  capability  requirements,  to:  conduct  analysis  and  evaluate 
alternatives  and  identify  preferred  solutions;  develop  and  approve  acquisition 
documents;  contract  using  all  available  statutory  and  regulatory  authorities  and 
waivers  and  deviations  of  such,  appropriate  to  the  situation;  identify  and  minimize 
technical  development,  integration,  and  manufacturing  risks;  and  rapidly  produce 
and  deliver  required  capabilities  (Joint  Chiefs  of  Staff,  2012). 

UONs  are  “capability  requirements  identified  by  a  DoD  component  as  impacting 

an  ongoing  or  anticipated  contingency  operation.  If  left  unfulfilled,  UONS  result  in 

capability  gaps  potentially  resulting  in  loss  of  life  or  critical  mission  failure”  (Joint  Chiefs 


of  Staff,  2012).  UONs  and  JUONs  are  required  to  be  revalidated  every  2  years  after  the 
original  validation  date  to  ensure  that  the  requirement  is  still  valid  and  to  faeilitate  the 
transition  to  an  enduring  requirement  or  the  assessment  of  limited  duration  sustainment 
(Joint  Chiefs  of  Staff,  2012). 

UONs  and  JUONs  are  required  to  have  an  “assessment  of  operational  utility  for 
the  capability  solution  within  90  days  of  the  initial  fielding”  (Joint  Chiefs  of  Staff,  2012). 
This  will  help  facilitate  the  movement  of  the  program  through  the  transition  and  to 
determine  its  sustainability.  There  are  three  assessment  categories:  Failure/Limited 
Success,  Success/Limited  Duration  Requirement,  and  Success/Enduring  Requirement 
(Joint  Chiefs  of  Staff,  2012). 

Prescribed  Rapid  Acquisition  Processes 

AFl  63-114  is  the  set  of  instructions  given  by  the  USAF  on  how  it  answers  UONs, 
JUONs  or  Chief  of  Staff  of  the  Air  Force  (AF/CC)  directions.  It  is  meant  to  provide  a 
framework  for  PMs  to  satisfy  the  urgent  needs  of  the  warfighter’s  to  reduce  the  capability 
gap  -defined  in  the  requirements  documentation.  A  program  is  designated  as  a  Quick 
Reaction  Capability  (QRC)  by  the  milestone  decision  authority  (MDA)  based  upon  the 
following  three  triggers,  with  an  expected  timeline  for  a  QRC  of  180  days  to  2  years 
(Department  of  the  Air  Force,  2011). 

1 .  Trigger  1  is  a  UON  given  by  a  Commander  Air  Force  Forces  (COMAFFOR)  such 
as  the  Commander  of  Air  Combat  Command  (COMACC). 

2.  Trigger  2  is  a  JUON  from  a  Unified  Combatant  Commander  such  as  the 
Commander  of  Central  Command  (CENTCOM)  or  Pacific  Command 

(P ACCOM).  The  JUON  will  be  validated  by  the  Joint  Requirements  Acquisition 
Cell  (JRAC)  and  passed  on  to  the  lead  service. 
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3.  Trigger  3  is  if  directed  by  the  AF/CC  to  “rapidly  fulfdl  a  validated  urgent 
operational  need”  (Department  of  the  Air  Force,  2011). 

The  designation  as  a  QRC  allows  the  programs  to  minimize  the  number  of 

reviews  that  are  required  and  provides  access  to  exemptions  and  waivers  not  normally 

given  to  traditional  acquisition  programs.  There  are  four  phases  for  a  QRC  after  its 

requirements  have  been  validated:  Course  of  Action  (COA)  Development,  Materiel 

Development  Decision  (MDD),  Execution,  and  Transition.  This  process  is  shown  below 

in  Figure  2. 


Requirement 

AF/CC 
Direction  or 
JUON 


Step1  Step  2 


COA 


UON 


CJCSI  3470.01  COA  QRC-MDD 

AFMO-601  Development 


step  3 


Develop  &  Field 
'  Source  funding 
'  Minimal  capability  fielded 
'  Rapid  contracting  methods 
'  Delegated  authorities 
'  Accept  risk 

'  Tailored  DT/OT  to  assess 
capabilities  &  limitations 
'  Warfighter  feedback 
'  Planned  sustainment 


Execution 


T  ransition 


Figure  2  QRC  Process  (Department  of  the  Air  Force,  2011) 

In  COA  development  the  PM  decides  on  which  of  the  different  possible  COAs 


that  the  program  will  follow.  During  the  MDD  the  proposed  solution  from  the  previous 


phase  is  validated  and  officially  chosen.  The  Execution  phase  is  where  the  bulk  of  the 


work  for  the  program  is  completed,  with  the  engineering  design,  testing  and  initial 


fielding  completed  during  this  phase.  The  Transition  phase  is  the  process  in  which  the 


program  is  either  transitioned  to  an  enduring  program  of  record,  sustained  in-theater  only. 


or  demilitarized  and  disposed  of.  (Department  of  the  Air  Eorce,  2011) 


An  important  aspect  for  the  QRC  programs  is  tailoring.  It  is  directed  that  the 


QRC  programs  will  use  an  expedited  review  process  along  with  streamlined 
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documentation  and  certifications  to  the  “maximum  extent  possible  and  accept  appropriate 
risk  to  provide  rapid  capability  to  war  fighting  commanders”  (Department  of  the  Air 
Force,  2011).  As  such,  if  it  is  not  a  statutory  requirement,  QRCs  will  most  likely  tailor 
regulatory  requirements  while  keeping  documentation  and  certifications  to  a  minimum. 
The  AFI  also  states  that  the  QRC  will  only  “provide  or  modify  the  minimum  number  of 
systems  needed  for  testing  and  in-theater  operations”  (Department  of  the  Air  Force, 
2011). 

Systems  Engineering 

The  International  Council  on  Systems  Engineering  (INCOSE)  defines  SE  as  “a 
discipline  that  concentrates  on  the  design  and  application  of  the  whole  (system)  as 
distinct  from  the  parts”  (Haskins,  Eorsberg,  &  Krueger,  2006).  Alternatively  the  DoD 
defines  SE  as  “integrating  technical  processes  to  define  and  balance  system  performance, 
cost,  schedule,  and  risk  within  a  family-of-systems  and  systems-of-systems  context” 
(Department  of  Defense,  2008).  The  Defense  Acquisition  Guidebook  (DAG)  defines  SE 
as  “is  a  methodical  and  disciplined  approach  for  the  specification,  design,  development, 
realization,  technical  management,  operations,  and  retirement  of  a  system”  (Defense 
Acquisition  University,  2004).  INCOSE  views  SE  as  a  collection  of  different  processes 
that  allow  the  optimization  of  a  complex  problem  set.  In  the  DoD  acquisition  world  SE 
has  evolved  into  multiple  Technical  Processes  and  Technical  Management  Processes. 

Eor  any  acquisition  program  in  the  DoD,  either  traditional  or  rapid,  the  PM  has 
the  responsibility  to  ensure  the  program  is  executed  properly,  instructions  and  laws  are 
followed,  establish  who  the  stakeholders  are  and  their  requirements,  coordinate  all 


11 


acquisition  and  SE  plans,  ensure  decision  are  documented,  and  manage  risk  (Defense 
Aequisition  University,  2004). 

The  Systems  Engineer  is  responsible  for  the  exeeution  of  the  SE  plan  ereated  with 
the  PM,  understanding  the  context  of  the  proposed  system  within  the  system-of-systems, 
assessing  process  improvements,  managing  the  technical  risks  of  the  program,  overseeing 
the  program’s  teehnieal  reviews,  ensuring  the  test  and  evaluation  master  plan  is  being 
followed,  and  reviewing  the  deliverables  from  contractors  (Defense  Aequisition 
University,  2004). 

Aceording  to  Defense  Aequisition  University  (DAU),  SE  ean  be  thought  of  as  16 
interrelated  processes,  categorized  as  either  teehnieal  processes  or  technical  management 
proeesses  as  seen  in  Table  1.  The  eight  teehnieal  proeesses  are  the  “top-down  design 
processes  and  bottom-up  realization  processes”  needed  to  take  a  user’s  needs  and  produce 
a  working  system.  This  is  contrasted  with  the  eight  teehnieal  management  proeesses 
which  “provide  insight  and  eontrol  to  assist  the  Program  Manager  and  Systems  Engineer 
to  meet  performance,  schedule,  and  cost  goals”  (Defense  Aequisition  University,  2004). 

Table  1  Systems  Engineering  Processes 


Technical  Processes 

Technical  Management  Processes 

Stakeholder  Requirements  Definition 

Technical  Planning 

Requirements  Analysis 

Decision  Analysis 

Architecture  Design 

Technical  Assessment 

Implementation 

Requirements  Management 

Integration 

Risk  Management 

Verification 

Configuration  Management 

Validation 

Technical  Data  Management 

Transition 

Interface  Management 
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Technical  Processes 


The  first  Technical  Process  to  discuss  is  the  Stakeholder  Requirements  Definition 
Process  in  which  the  PM  will  “elicit,  negotiate,  document  and  maintain  stakeholders’ 
requirements  for  the  system-of-interest  within  a  defined  environment”  (Haskins  et  ah, 
2006).  The  Stakeholder’s  Requirements  Definition  Process  allows  the  designated  lead 
office  to  work  with  the  program  stakeholders  to  define  the  requirements  for  the  system 
and  translate  those  system  level  requirements  into  technical  requirements  (Defense 
Acquisition  University,  2004).  The  user  requirement  typically  requires  refinement  by  the 
acquisition  program  office  so  that  the  overall  program  can  be  scoped  and  be  managed  to 
balance  user  needs  and  system  performance  with  schedule  and  cost.  This  process  ensures 
that  the  different  stakeholders  all  have  a  say  in  the  system  definition  and  agree  on  the 
future  vision  of  what  the  system  will  be  capable  of  doing.  This  process  helps  to 
complement  the  Architecture  Design  Process  and  the  Requirements  Analysis  Process  by 
reducing  the  chance  of  requirements  creep  and  a  change  in  focus  of  the  system  (Defense 
Acquisition  University,  2004). 

The  next  process  is  the  Requirements  Analysis  Process  where  the  PMs  “review, 
assess,  prioritize,  and  balance  all  stakeholder  and  derived  requirements  (including 
constraints),  and  to  transform  those  requirements  into  a  functional  and  technical  view  of  a 
system  description  capable  of  meeting  the  stakeholders’  needs.”  This  decomposition  of 
the  stakeholders’  requirements  into  system  specifications  allows  for  the  system  to  be 
designed  without  introducing  “implementation  biases”  (Haskins  et  ah,  2006).  During  the 
beginning  of  the  program  the  process  is  used  in  concert  with  the  Stakeholder’s 
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Requirements  Definition  Process  to  define  what  the  system  will  be  required  to  do,  but  as 
the  program  matures  and  the  design  becomes  more  defined  the  process  should  “support 
allocation  and  derivation  of  requirements  down  to  the  system  elements  representing  the 
lowest  level  of  the  design”  (Defense  Acquisition  University,  2004). 

INCOSE  views  the  Architecture  Design  Process  as  the  creation  of  a  “system 
architecture  baseline  that  satisfies  the  requirements”  (Haskins  et  ah,  2006).  Another  view 
is  that  the  Architecture  Design  Process  allows  the  PM  and  SE  to  “translate  the  outputs  of 
the  Stakeholder  Requirements  Definition  and  Requirements  Analysis  Processes  into 
alternative  design  solutions”  (Defense  Acquisition  University,  2004).  This  architecture  is 
used  to  examine  any  configuration  changes  that  are  brought  up  in  the  design  process  and 
to  ensure  that  system  interfaces  have  been  discussed.  The  Architecture  Design  Process, 
along  with  the  Stakeholder  Requirements  Definition  and  Requirements  Analysis, 
combine  to  provide  insights  into  technical  risks  along  with  mitigation  strategies  for  the 
program.  Defining  and  analyzing  the  architecture  during  this  process  allows  the  PM  and 
SE  to  look  at  concepts  such  as  maintainability,  sustainability,  performance  and  cost 
before  finalizing  the  expected  design  (Defense  Acquisition  University,  2004). 

The  Implementation  Process’s  purpose  is  “to  design,  create  or  fabricate  a  system 
element  conforming  to  that  element’s  detailed  description”  (Haskins  et  ah,  2006).  That  is 
to  say  that  the  Implementation  Process  is  when  the  different  parts  of  the  user’s  product 
are  physically  created.  There  are  two  phases  for  the  Implementation  Process:  design  and 
realization.  The  design  phase  includes  the  engineering  and  contracting  activities  to 
develop  the  “detailed  design  down  to  the  lowest  level  system  elements  in  the  system 
architecture”  (Defense  Acquisition  University,  2004).  The  realization  phase  of  the 
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Implementation  Proeess  is  “the  proeess  of  building  the  system  elements  using  speeified 
materials  and  fabrieation  and  produetion  tools/proeedures  identified  during  the  design” 
(Defense  Aequisition  University,  2004).  This  eould  inelude  manufaeturing  or  eoding  the 
part  to  meet  all  the  speeifieations  spelled  out  in  the  previous  proeesses. 

The  Integration  Proeess  is  how  the  subsystems  ereated  during  the  Implementation 
Proeess  eonneet  together  to  form  the  full  system.  It  combines  all  of  the  individual  parts 
to  “realize  the  system-of-interest  [...]  in  accordance  with  the  architectural  design 
requirements  and  the  integration  strategy”  (Haskins  et  ah,  2006).  This  is  an  iterative 
process  where  all  of  the  design  considerations  will  be  implemented  to  ensure  that  the 
different  parts  of  the  system  all  correctly  fit  together  to  meet  the  purpose  of  the  user. 

This  works  in  concert  with  the  Verification  process  to  ensure  that  each  part  and 
subsystem  meets  the  requirements  for  it.  The  Interface  Management  Process  helps 
ensure  that  each  subsystem  is  able  to  connect  to  the  correct  mate  and  that  the  system  as  a 
whole  is  able  to  connect  to  other  systems  as  required  for  the  capability  being  provided 
(Defense  Acquisition  University,  2004). 

The  Verification  and  Validation  Processes  include  SE  activities  in  which  the  PM 
verifies  that  the  requirements  are  being  addressed  in  the  design  and  then  validates  that  the 
product  produced  meets  the  requirements  of  the  user  (Haskins  et  ah,  2006).  The 
Verification  Process  ensures  that  each  “system  element  performs  its  intended  functions 
and  meets  all  performance  requirements  listed  in  the  system  performance  specification” 
(Defense  Acquisition  University,  2004).  In  other  words,  verification  ensures  that  what 
was  built  was  done  correctly.  This  can  be  done  by  a  combination  of  demonstration, 
examination,  analysis,  and  testing.  The  Validation  Process  is  the  way  that  the  PM  and  SE 
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can  prove  that  the  system  built  is  correet  for  the  needs  stated  by  the  stakeholders.  If  the 
Verification  process  asks  “did  we  build  what  we  wanted  to,”  then  the  Validation  process 
asks  “did  we  build  what  we  needed”  (Defense  Aequisition  University,  2004)?  This 
proeess  consists  of  evaluations  that  examine  the  system  for  operational  suitability, 
effectiveness  sustainability  and  survivability  under  realistie  environmental  constrains. 

The  Transition  Proeess ’s  purpose  is  “to  transfer  custody  of  the  system  and 
responsibility  for  system  support  from  one  organizational  entity  to  another”  (Haskins  et 
ah,  2006).  The  Transition  Process  is  how  the  system  will  be  delivered  to  the  end-user. 
This  ineludes  training  personnel  to  use  the  system,  the  installation  of  the  system  and  the 
delivery  of  any  manuals  or  teehnieal  data  to  the  correet  stakeholder.  The  Transition 
Process  begins  early  in  the  development  of  the  system  to  allow  for  proper  transitioning  of 
the  system  and  ineludes  maintenanee  and  support  functions  for  the  entire  system  under 
design  (Defense  Aequisition  University,  2004).  This  is  a  crucial  step  in  the  aequisition 
proeess  as  this  is  when  the  program  is  turned  over  to  the  user  to  be  implemented  in  the 
field,  and  in  the  ease  of  the  DoD  this  is  when  warfighters’  lives  could  be  at  stake 
depending  on  how  well  the  system  is  designed. 

Technical  Management  Processes 

The  first  of  the  teehnieal  management  proeesses  (TMPs)  is  Teehnieal  Planning 
which  includes  “defining  the  scope  of  the  teehnieal  effort  required  to  develop,  field,  and 
sustain  the  system,  as  well  as  providing  eritical  quantitative  inputs  to  program  planning 
and  life-cycle  cost  estimates”  (Defense  Aequisition  University,  2004).  Technical 
planning  allows  the  Systems  Engineer  and  the  PM  to  plan  for  and  program  money  for 
different  planned  activities  along  with  helping  to  ereate  a  foundation  for  the  risk 


16 


management  process  and  the  creation  of  the  measures  that  will  be  used  during  the 
Technical  Assessment  Process  (Defense  Acquisition  University,  2004).  This  process  is 
continually  re-evaluated  at  each  phase  of  the  acquisition  program. 

The  Decision  Analysis  Process  is  the  way  that  the  DAG  defines  the  decision 
making  process  to  allow  for  traceability  of  decisions  along  with  the  creation  of  an 
actionable  plan  (Defense  Acquisition  University,  2004).  It  has  multiple  levels,  with 
multiple  lower  level  discrete  analyses’  being  “aggregated  into  a  higher-level  view 
relevant  to  the  decision  maker  and  other  stakeholders”  (Defense  Acquisition  University, 
2004).  This  process  should  influence  and  interact  with  other  SE  processes  including: 
Technical  Planning,  Technical  Assessment,  Stakeholder  Requirements  Definitions, 
Requirements  Analysis,  and  Architecture  Design  (Defense  Acquisition  University,  2004). 

By  conducting  the  Technical  Assessment  Process  the  Systems  Engineer  is  able  to 
“compare  achieved  results  against  defined  criteria  to  provide  a  fact-based  understanding 
of  the  current  level  of  product  knowledge,  technical  maturity,  program  status,  and 
technical  risk”  (Defense  Acquisition  University,  2004).  This  process  allows  the  PM  to 
have  access  to  data  to  conduct  decisions  about  the  program.  It  is  conducted  throughout 
the  life-cycle  of  the  program  and  provides  the  data  necessary  to  make  any  corrections 
needed  for  the  program. 

The  Requirements  Management  process  ensures  that  the  program  turns  out  a 
capability  or  item  that  meets  the  needs  of  the  end  user  (Defense  Acquisition  University, 
2004).  Those  needs  are  normally  defined  during  the  Stakeholder  Requirements 
Definition  process  along  with  the  Decision  Analysis  Process  and  are  updated  as  required 
for  changes  provided  by  the  stakeholder  (Defense  Acquisition  University,  2004).  This 
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process  also  allows  for  the  traceability  of  high  level  requirements  to  detailed  design 
specifications  and  vice-versa.  By  allowing  a  two-way  traceability  it  ensures  that  no  detail 
specifications  are  orphaned  from  system  needs  nor  are  any  system  needs  not  meet  during 
the  design  (Defense  Acquisition  University,  2004). 

The  Risk  Management  Process  is  “the  overarching  process  that  encompasses 
identifieation,  analysis,  mitigation  planning,  mitigation  plan  implementation,  and  tracking 
of  program  risk”  (Defense  Acquisition  University,  2004).  Risk  is  defined  as  “an 
unwanted  event  that  may  or  may  not  occur  in  the  future”  and  needs  to  be  managed  at  all 
phases  of  the  program  (Defense  Aequisition  University,  2004).  This  process  allows  the 
PM  and  SE  to  manage  the  program  and  minimize  the  programmatie  and  technical  risk  of 
the  program. 

Configuration  management  is  more  than  ensuring  the  output  of  the  program  is  in 
controlled  versions  for  upgrades,  it  “allows  technical  insight  into  all  levels  of  the  system 
design  and  is  the  principal  methodology  for  establishing  and  maintain  consistency  of  a 
system’s  functional,  performance,  and  physical  attributes  with  its  requirements,  design, 
operational  information  throughout  the  system’s  life  cycle”  (Defense  Acquisition 
University,  2004).  While  the  processes  is  ongoing  during  all  phases  of  the  program  it  is 
important  that  the  different  baselines,  such  as  the  functional  and  allocated  baselines,  are 
used  in  ensuring  the  correct  configuration  is  being  worked  on  by  the  program  team. 

AFRL  Systems  Engineering 

AFRLI  61-104  defines  how  the  Air  Force  Researeh  Faboratory  (AFRF)  PM  and 
scientists  look  at  SE  with  respect  to  reviewing  and  executing  programs  under  their 
purview.  It  is  derived  from  the  16  processes  defined  in  the  DAG  and  should  be  tailored 
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for  the  particular  projects  and  programs  that  are  being  conducted  in  the  lab.  The  process 
described  in  AFRLI  61-104  is  how  AFRL  “[decomposes]  scientific  research  objectives  to 
knowledge,  or  capability  needs  to  technology  alternatives”  (AFRL,  2013b).  The  below 
figure  shows  how  AFRL  views  their  process  and  incorporates  the  8  Technical 
Management  Processes  and  8  Technical  Processes  defined  in  the  DAG. 


Customer 

Requirements 


Technology 

Alternatives 


ITERATIVE 


Technology 

Demonstration 


Figure  3  S&T  Systems  Engineering  Process  (AFRL,  2013b) 

The  AFRLI  recommends  that  the  8  TMPs  be  conducted  “continuously  and 
concurrently  while  the  eight  technical  processes  are  performed  sequentially,  although 
with  considerable  iteration  and  feedback  checking”  (AFRL,  2013b). 

The  AFRLI  also  lists  eight  questions  that  it  expects  its  PMs  and  SEs  to  use  during 
the  assessment  of  their  programs.  These  questions  were  derived  from  the  16  DAG 
processes. 

1 .  Who  is  your  customer? 

2.  What  are  the  customer’s  requirements? 

3.  How  will  you  demonstrate  you  have  met  the  requirements? 

4.  What  are  the  technology  options? 

5.  Which  is  the  best  approach? 

6.  What  are  the  risks  to  developing  the  selected  technology? 
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7.  How  will  you  structure  your  program  to  meet  requirements  and  mitigate  risk? 

8.  What  is  your  business-based  transition  plan  that  meets  customer  approval? 

As  ean  be  expected,  the  eight  questions  tie  in  with  the  16  processes  described  in 

the  DAG  and  are  used  to  “assess  the  sufficiency  of  the  SE  process  on  a  particular  S&T 
program”  (AFRL,  2013b). 

Tailoring 

The  need  for  tailoring  is  paramount  in  rapid  aequisition,  not  only  tailoring  the 
aequisition  processes  used  but  also  the  SE  activities  eompleted,  and  the  tailoring  should 
“reflect  the  system’s  maturity  and  complexity,  size  and  scope,  [and]  life-cycle  phase” 
(Defense  Aequisition  Elniversity,  2004).  Rapid  aequisition  is  often  tailored  due  to  the 
smaller  seope  and  less  complex  solutions  that  are  required  to  meet  the  expedited 
timelines.  SE  processes  are  normally  designed  “with  the  mindset  of  developing  a 
eompletely  new  complex  system”  (Pickard  &  Nolan,  2010).  Pickard  and  Nolan 
recommend  using  Risk  Management  and  Probability  Calculus  to  determine  whieh 
proeesses  need  to  be  completed  and  to  what  level  of  rigor.  Risk  Management  is  the  SE 
process  used  to  identify  and  reduce  uncertainty  (Pickard  &  Nolan,  2010).  Probability 
Calculus  is  the  comparison  of  the  cost  of  preventive  measures  versus  the  probability  of 
harm  multiplied  by  the  loss.  If  the  cost  is  less  than  the  product  of  the  harm  and  loss  then 
the  preventative  measure  should  be  included,  and  if  the  eost  is  greater  then  it  should  be 
excluded.  In  the  context  of  rapid  aequisition  tailoring,  engineering  design  and  all  of  the 
SE  and  aequisition  processes  used  can  be  eonsidered  a  risk  mitigation  proeess  whereby 
“every  requirements  speeifieation,  every  arehitecture,  every  drawing,  every  analysis  and 
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every  test  is  aimed  at  redueing  the  risk  that  the  solution  will  not  be  fit  for  purpose” 
(Piekard  &  Nolan,  2010). 

Pickard  and  Nolan  focus  on  two  types  of  tailoring:  “a  decision  about  whether  to 
apply  a  process,  and  the  second  is  to  choose  between  two  alternative  processes”  (Pickard 
&  Nolan,  2010).  The  first  is  examining  which  process  to  include  or  exclude  while  the 
second  is  determining  which  of  two  processes  to  include  when  they  both  will  meet  a 
certain  need  or  requirement.  They  found  the  introduction  of  risk  into  the  system  in  a 
controlled  manner  is  acceptable,  with  an  understood  trade-off  in  the  value  of  the  system. 
They  do  give  one  caveat  on  where  not  to  tailor  a  program,  safety  critical  systems.  As  the 
probability  of  occurrence  is  defined,  such  as  1  failure  in  10,000  hours  of  usage,  “all 
mitigations  required  to  achieve  this  level  of  probability  of  occurrence  have  to  be  applied 
and  cannot  be  tailored  out”  (Pickard  &  Nolan,  2010). 

Beasley  and  Partridge  discuss  the  fact  that  optimizing  each  subsystem  does  not 
mean  you  are  optimizing  the  overall  system;  the  focus  needs  to  be  on  “trying  to  make  the 
best  system  it  can  [be]”  (Beasley  &  Partridge,  2010).  This  focus  can  help  alleviate  the 
sub-optimization  of  the  overall  project  by  optimizing  a  sub-activity  or  process.  Each 
process  must  work  in  harmony  with  the  others  so  that  the  goal  of  an  optimized  system 
can  be  achieved. 

Previous  Research 

The  study  completed  by  Capt  Kipp  Johnson  looked  at  the  rapid  acquisition  case 
study  of  the  Self-Awareness  Space  Situational  Awareness  (SASSA)  Program  and  how 
that  program  used  a  tailored  versus  DoD  prescribed  Systems  Engineering  process.  The 
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author  found  that  while  the  program  deviated  from  standard  SE  proeesses,  not  all  of  the 
changes  were  beneficial  to  the  programs  performance,  schedule  and  cost  (Johnson,  2010). 
He  found  that  by  exempting  SASSA  from  the  JCIDS  process  the  program  was  able  to 
“move  more  quickly  than  a  JCIDs  program”  while  also  running  the  risk  that  the  final 
output  of  the  program  might  not  meet  the  user’s  needs  (Johnson,  2010). 

Another  study  looked  at  different  principles  of  rapid  acquisition  to  determine  how 
the  systems  engineering  process  could  be  tailored.  This  was  done  by  interviewing  the 
senior  leaders  for  a  number  of  AFRL  programs  and  creating  a  framework  to  define  the 
level  of  rigor  that  the  different  systems  engineering  processes  should  be  completed  to. 
Their  findings  and  associated  framework,  while  helpful  to  a  program  manager  in  a 
holistic  sense  at  AFRL,  is  not  generalizable  to  non  AFRL  projects  and  programs  (Behm, 
Pitzer,  &  White,  2009). 

One  of  the  key  research  questions  postulated  by  Smith  (2011)  was  “what  accepted 
activities  in  rapid  development  literature  and  practice  correlate  to  Defense  Acquisition  SE 
activities”  (Smith,  2011).  His  analysis  of  the  literature  showed  that  stakeholders’ 
requirements  definition,  architecture  design  and  technical  planning  were  all  emphasized. 
This  was  completed  using  a  qualitative  analysis  of  the  literature  and  focused  interviews 
with  leaders  in  AFRLs  core  process  programs  administering  rapid  development  programs 
trying  to  deliver  new  technologies  inside  of  two  years.  While  this  framework  states  a 
qualitative  view  that  these  processes  are  the  most  important  it  does  not  go  into  detail  on 
the  level  of  tailoring  that  best  suits  different  projects  or  how  they  interact  with  other 
processes  to  create  a  successful  program. 
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In  2012  AFIT’s  Systems  Engineering  Researeh  Center  (SERC)  published  its 
report  on  Expedited  Systems  Engineering  for  Rapid  Capability  and  Urgent  Needs  which 
discussed  its  findings  on  the  different  ways  that  rapid  acquisition  can  be  completed.  It 
makes  recommendations  based  upon  three  areas:  1)  organizational  best  practices;  2)  “go 
fasf  ’  cultural  best  practices;  and  3)  “rapid  world”  best  practices. 

They  found  that  “rapid  requires  an  integrated  approach:  People  making 
judgments,  Processes  for  task  reductions,  and  Product  aspects  focused  on  rapid 
objectives”  (Eepore  &  Colombi,  2012).  When  looking  at  the  organizational  best 
practices  with  respect  to  this  thesis,  the  report  recommend  the  use  of  mature  technology 
and  “focus  on  the  state  of  the  possible”  (Eepore  &  Colombi,  2012).  The  authors 
recommended  using  a  stable  requirement  list  gathered  from  the  customer  while  using  an 
incremental  development  process  for  the  system  under  design.  Other  recommendations 
included  the  acceptance  of  some  risk  and  trying  to  exploit  any  flexibility  allowed  (Eepore 
&  Colombi,  2012). 

The  findings  for  cultural  best  practices  include  the  use  of  “intense  and  efficient 
knowledge  sharing  [. . .]  to  enable  stabilization  and  synchronization  of  information” 
(Eepore  &  Colombi,  2012).  One  other  important  recommendation  at  the  “rapid  world” 
level  is  that  the  DoD  should  focus  not  on  having  a  single  rapid  organization,  but  many 
flexible  rapid  development  teams  with  a  shared  knowledge  base  (Eepore  &  Colombi, 
2012). 

Summary 

This  section  discussed  how  the  DoD  views  SE  and  what  has  been  previously 
researched.  It  has  showcased  the  different  technical  and  technical  management  processes 
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incorporated  into  the  larger  SE  proeess,  while  also  laying  the  framework  for  the  researeh 
questions  that  this  thesis  addresses. 
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III.  Methodology 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  explain  the  methodology  used  to  understand  the 
acquisition  and  SE  processes  used  by  the  United  States  Air  Force  to  complete  rapid 
acquisition  and  how  those  programs  were  tailored  to  meet  the  expedited  schedule 
requirements.  There  are  five  research  questions  investigated  during  this  research: 

1 .  What  processes  does  the  United  States  Air  Force  use  to  complete  rapid  acquisition 
projects  and  programs? 

2.  Are  the  observed  processes  consistent  with  prescribed  instructions? 

3.  What  SE  activities  are  used  in  rapid  acquisition  programs  in  the  United  States  Air 
Force? 

4.  How  are  rapid  acquisition  programs  tailored  in  the  United  States  Air  Force? 

5.  Which  program  attributes  are  used  to  determine  program  tailoring? 

This  research  was  completed  in  a  four  step  process  based  upon  the  qualitative 
research  design  described  by  Merriam,  in  which  the  first  phase  is  the  literature  review, 
followed  by  purposeful  sampling  and  data  collection.  The  third  phase  is  the  analysis  of 
the  collected  data,  and  the  fourth  and  final  phase  is  drawing  conclusions  with  respect  to 
the  research  questions  (Merriam,  2009).  Figure  4  shows  the  methodology  process  used 
during  the  study. 
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Literature  Review 

•  In  Depth  analysis 
Definition  of  basic  processes 
•  Review  of  DODI  &  APIs 
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Purposeful  Sampling  and 
Data  Collection 

12  PM  /  SE  interviewed 
Open  ended  questions 
Initial  Interviews  feed  into  later 
interviews 


Conclusions 

•  Pull  salient  points  from  data 
set 

•  Synthesize  themes  from  data 
analysis  with  literature  review 

•  Recommendations 


Data  Analysis 

•  Interviews  coded  /  categorized 
during  the  process 

•  Continued  until  allowable  time 
elapsed 
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Figure  4  Qualitative  Methodology  Proeess 

Setting 

This  is  a  qualitative  study  examining  how  the  Air  Foree  eompletes  rapid 
acquisition.  The  interviews  were  completed  in  two  locations:  face  to  face  at  AFIT  and 
over  the  phone  while  the  interviewees  were  at  their  work  locations.  The  location  at  AFIT 
allowed  for  a  quiet  situation  with  little  to  no  distractions  for  the  interviewee.  The  phone 
interviews  were  conducted  to  minimize  the  disruption  to  the  interviewee’s  work  and  to 
facilitate  the  interviewing  of  personnel  who  were  not  located  at  Wright-Patterson  AFB. 


Participants 

The  SMEs  themselves  were  selected  because  they  are  acquisition  personnel  who 
have  experience  in  the  rapid  acquisition  processes.  Due  to  the  small  population  of 
program  managers  with  rapid  acquisition  experience  and  the  time  frame  associated  with 
this  research,  the  number  of  interviews  was  kept  to  twelve.  The  sampling  technique  used 
in  this  thesis  was  a  non-probabilistic  purposive-based  sampling  where  initial  SMEs  were 
selected  based  upon  personal  recommendations  from  the  research  committee.  Then,  the 
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SMEs  were  asked  to  reeommend  others  that  they  have  worked  with  that  had  the 
neeessary  background  to  be  included  in  the  study.  This  type  of  snowball  sampling  helps 
to  reach  underserved  or  hard  to  reach  populations  such  as  rapid  acquisition  SMEs 
required  for  this  thesis  (Eund  Research  Ltd,  2012).  As  mentioned,  due  to  time  constraints 
associated  with  the  research  program,  the  total  number  interviews  conducted  was  capped 
at  twelve  to  allow  proper  time  to  conduct  data  analysis  and  to  draw  conclusions. 

As  mentioned  previously,  the  participants  were  selected  as  SMEs  with  experience 
in  rapid  acquisitions  within  the  Air  Eorce.  These  participants  were  required  to  have  been 
associated  with  rapid  acquisition  programs  and  to  have  knowledge  and  understanding  of 
how  they  were  conducted  and  what  processes  were  used.  Of  the  twelve  participants,  all 
were  members  of  the  Air  Eorce;  nine  were  civilian  employees,  two  were  Officers  and  one 
was  a  contractor.  Two  had  reported  spending  a  portion  of  their  career  at  a  systems 
program  office  (SPO),  with  five  having  spent  time  working  in  AFRL.  Due  to  the  need 
for  the  respondents  to  be  experts  in  their  fields,  many  of  the  participants  held  a  senior 
level  position  inside  their  respective  organizations  with  nine  being  considered  senior 
(equivalent  of  government  service  (GS)  level  14-15),  two  mid-level  (GS  level  12-13),  and 
one  contractor  (AFRL,  2011).  The  seniority  level  of  the  SMEs  is  shown  below  in  Figure 
5. 
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Senior  Middle  Jiimoi  Contiactoi 


(GS  1 4- 1 5)  (GS  12-13)  (<GS- 1 1 ) 

Seniority  Level 


■  Rapid  SPO 
□  AFRL 

■  Traditional  SPO 


Figure  5  Seniority  Level  of  SMEs 

The  SMEs  interviewed  for  this  study  had  different  baekgrounds  and  experiences 
with  rapid  acquisition  and  the  acquisition  process  as  a  whole.  Eive  personnel  work  in 
AERE  on  rapid  development  projects  in  various  locations,  while  another  two  work  as 
senior  leadership  at  one  of  the  laboratory  directorates  and  will  be  referenced  as  lab 
personnel  for  the  duration  of  this  thesis.  Two  personnel  work  in  traditional  program 
management  positions  in  program  offices  at  AEECMC  and  will  be  referenced  as 
Traditional  SPO  personnel.  Another  two  SMEs  work  at  an  organization  focused  on  rapid 
design  and  prototyping  which  is  managed  by  AEECMC.  The  final  interviewee  was  a  PM 
at  an  office  that  works  on  sensitive  rapid  acquisition  for  the  intelligence  community. 
These  final  three  SMEs  are  designated  as  Rapid  SPO  personnel  due  to  the  uniqueness  of 
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their  programs  with  respect  to  the  acquisition  corps  as  a  whole.  The  distribution  of 
personnel  interviewed  can  be  seen  below  in  Figure  6. 


■  Traditional  SPO 
□  AFRL 

■  Rapid  PC) 


Figure  6  Personnel  Distribution 

Measurement  Instruments 

To  collect  the  data  from  the  participants,  a  semi-structured  interview  was 
conducted  to  elicit  responses.  The  interviewees  were  instructed  that  they  would  be  asked 
sixteen  questions  and  they  did  not  have  to  answer  any  or  all  of  the  questions.  A  copy  of 
the  interview  protocol  used  during  the  interviews  is  included  in  Appendix  A. 

The  purpose  of  the  interviews  was  to  gather  knowledge  from  the  different  SMEs 
to  understand  the  different  processes  used  across  the  Air  Force.  Interviewing  the  SMEs 
allowed  the  researcher  to  gather  data  from  across  many  programs  but  to  keep  the 
sensitive  nature  of  the  programs  at  bay  as  they  were  not  discussed  in  any  detail  that  might 
compromise  the  programs  or  this  research.  The  information  was  recorded  in  all  of  the 
interviews  except  one,  so  that  the  data  could  be  transcribed  and  then  coded  during  the 
data  analysis  phase.  Eor  the  one  interview  that  wasn’t  recorded,  notes  were  taken  and 
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then  reviewed  by  the  interviewee  to  ensure  that  the  answers  were  100%  faetual  to  what 
was  discussed  during  the  interview.  Another  outlier  was  interview  #10  in  which  the 
interviewee  brought  a  second  SME  to  the  interview.  Their  responses  are  combined  in 
Appendix  D,  Summary  of  Interview  9. 

The  interview  questions  were  created  specifically  to  answer  the  research  questions 
of  this  thesis.  The  purpose  of  the  interview  questions  was  to  elicit  responses  from  the 
interviewees  with  regard  to  their  experience  with  rapid  acquisition  in  the  Air  Force.  The 
interviews  ranged  from  30  minutes  to  an  hour  and  a  half  depending  on  the  respondent’s 
comments  and  the  need  for  follow  up  questions  from  the  interviewer  for  clarification  of 
any  answers.  The  questions  were  sent  to  all  of  the  interviewee’s  before  the  interview  to 
allow  them  to  familiarize  themselves  with  the  content  of  the  interview  and  gather  any 
information  they  would  need  to  answer  the  questions.  An  attachment  was  also  sent  to 
each  interviewee  that  explained  the  eight  SE  management  processes  and  eight  technical 
management  processes  as  defined  by  the  Defense  Acquisition  Guidebook  (DAG)  as  seen 
in  Appendix  B.  This  attachment  also  included  nominal  sub  processes  that  one  would 
expect  to  complete  with  respect  to  the  sixteen  SE  processes  as  culled  from  the  thesis  of 
Maj  Behm,  Maj  Pitzer  and  Ms.  White  (Behm,  Pitzer,  &  White,  2009). 

Validity  and  Reliability 

Validity  is  “the  extent  to  which  the  instrument  measures  what  it  was  intended  to 
measure”  (Bui,  2014).  The  interview  script  was  designed  specifically  for  this  research 
and  it  was  reviewed  by  SE  and  PM  experts  to  ensure  that  the  questions  being  asked 
would  result  in  the  answers  that  were  applicable  to  the  research.  Another  aspect  of 
validity  comes  from  data  triangulation,  which  refers  to  taking  a  broad  sampling  of  data 
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from  multiple  collection  points  as  was  done  here,  i.e.  interviewing  personnel  from 
multiple  Air  Force  agencies.  Data  collection  from  multiple  people  and  agencies  helps  to 
raise  the  internal  validity  of  the  research  because  it  reduces  bias  from  any  one  viewpoint 
(Merriam,  2009). 

Procedure 

The  data  was  collected  through  semi-structured  interviews.  As  discussed  in  the 
Setting  section,  the  interviews  were  conducted  both  face-to-face  and  over  the  phone  due 
to  travel  and  time  limitations.  When  the  interviews  were  conducted  in  person,  they  were 
completed  at  the  AFIT  campus  in  a  room  free  of  interruptions  and  distractions.  When 
conducting  the  phone  interviews  the  interviewee  was  at  their  work  desk.  This  allowed 
the  interviewee  to  feel  comfortable  and  secure  in  their  surroundings.  Before  each 
interview,  the  interviewer  would  complete  some  short  personal  discussion  with  each 
interviewee  to  put  the  participant  at  ease  and  to  build  rapport.  At  the  start  of  each 
interview,  the  interviewer  would  ask  if  the  interviewee  would  allow  the  interview  to  be 
taped  and  transcribed  for  data  analysis  purposes.  Each  interview  then  began  with  a 
reading  of  a  preamble  to  remind  the  interviewee  of  the  subject  that  was  to  be  discussed. 
The  interviewer  would  then  pose  each  question  to  the  interviewee  in  turn,  asking  follow¬ 
up  questions  as  needed,  as  shown  in  the  interview  protocol  in  Appendix  A.  After  the 
interview  was  completed  each  taped  interview  was  transcribed  using  a  denaturalism 
methodology  that  removes  “idiosyncratic  elements  of  speech  (e.g.  stutters,  pauses, 
nonverbals,  involuntary  vocalizations)”  (Oliver,  Serovich,  &  Mason,  2005).  The  use  of  a 
verbatim  transcript  was  used  to  minimize  investigator  bias  before  handling  and 
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interpreting  the  data  (Haleomb  &  Davidson,  2006).  Eaeh  question  posed  during  the 
interview  was  mapped  to  one  of  the  researeh  questions,  as  shown  below  in  Table  2. 

Table  2  Interview  Questions 


Interview  Question 

Research  Question 

1 .  What  experience  do  you  have  with  rapid  acquisitbn? 

N/A 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

1 

3.  Does  your  oflice  folbw  the  QRC  process  defined  in  AFI 63- 1 14? 

2 

4.  How  do  these  programs  begin  (i.e.  iniiation  by  UON/JUON,  technobgy  push.)? 

2 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  oflice  uses? 

2 

6.  Do  you  view  rapb  acquisitbn  as  an  mcremental  process  or  a  smgle  time 
solution? 

2 

7.  What  SE  activities  did  your  programs  mcbde? 

3 

8.  How  did  you  decide  which  processes  to  bclude? 

3 

9.  How  iterative  are  the  SE  activities  used  b  your  programs? 

3 

10.  What  mteractions  did  you  see  between  the  SE  process  mcbded  or  excluded? 

3 

1 1 .  How  have  projects  you  have  been  mvolved  m  tailored  the  acquisition  process? 

4 

1 2.  How  do  you  determbe  to  what  level  a  program  needs  to  be  tailored? 

4 

13.  What  eflects  did  tailoring  have  on  the  overall  project? 

4 

1 4.  When  determinbg  how  to  tailor  a  program,  do  you  start  at  a  rninirnum  basefine 
and  add  activities  or  do  you  start  with  a  standard  'whole'  program  and  remove 
activities? 

4 

1 5.  What  attributes  does  your  organization  use  to  determbe  how  a  program  is 
tailored? 

5 

1 6.  What  bteractions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

5 

Data  Analysis 

The  data  analysis  eonsisted  of  four  phases:  open  eoding,  analytieal  eoding, 
eategory  eonstruetion,  and  drawing  eonelusions.  Open  eoding  is  deseribed  as  the  taking 
of  notes  based  upon  data  eolleeted  from  the  SME  interviews.  These  are  the  researeher’s 
thoughts  of  what  the  data  is  deseribing  and  are  not  limited  to  preeoneeived  eoneepts 
(Merriam,  2009).  Eaeh  transcript  is  analyzed  through  the  open  coding  process  and  has 
notes  describing  what  the  key  thoughts  and  ideas  are.  These  notes  were  placed  on 
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printouts  of  each  transcript  and  then  recorded  in  a  Microsoft  Excel  file  that  annotated  the 
interview  number,  page  and  line  number  of  the  data  along  with  the  code. 

The  second  phase  is  analytical  coding  which  is  described  as  the  grouping  of  the 
open  codes  (Merriam,  2009).  In  this  phase  the  codes  themselves  are  interpreted  and 
grouped  based  upon  the  meanings  of  the  data.  This  was  completed  in  the  Excel  file  by 
grouping  each  data  point  with  others  that  shared  common  points  or  themes.  These  groups 
lead  to  the  next  phase  of  the  data  analysis. 

The  third  phase  is  the  construction  of  the  different  categories  based  upon  the 
analytical  coding  of  phase  two.  The  categories  are  populated  by  the  data  points  that  are 
culled  from  the  analytical  coding  based  upon  patterns  and  any  commonality  found.  Each 
category  was  analyzed  and  modified  as  more  of  the  interview  data  was  incorporated  into 
the  pool  of  analyzed  data.  The  categories  had  five  criteria  that  they  had  to  meet  before 
they  could  be  considered  as  a  final  category  for  the  research:  “be  responsive  to  [. . .]  the 
research  questions,  be  as  sensitive  to  the  data  as  possible,  be  [collectively]  exhaustive,  be 
mutually  exclusive,  [and]  be  conceptually  congruent”  (Merriam,  2009). 

Examining  the  five  criteria  further  we  see  that  responsiveness  means  that  each 
category  should  somehow  be  related  to  and  answer  one  of  the  research  questions 
purposed  by  this  thesis  (Merriam,  2009).  Sensitive  categories  should  be  named  in  such  a 
way  that  “an  outsider  should  be  able  to  read  the  categories  and  gain  some  sense  of  their 
nature”  (Merriam,  2009).  Exhaustive  means  that  all  of  the  relevant  data  is  placed  into 
one  of  the  categories  while  mutually  exclusive  means  that  each  relevant  data  point  is 
place  only  able  to  be  placed  in  a  single  category  (Merriam,  2009).  The  final  criterion, 
conceptually  congruent,  means  “that  the  same  level  of  abstraction  should  characterize  all 


33 


categories  at  the  same  level”  (Merriam,  2009).  The  final  eategories  and  eodes  used 
during  the  study  will  be  discussed  further  in  Chapter  4. 

The  final  phase  of  the  data  analysis  is  the  drawing  of  conclusions  that  answer  the 
research  questions.  This  is  done  hy  examining  each  of  the  categories  that  were 
created  to  describe  the  data  collected  and  then  pulling  the  salient  points  and  themes 
out.  Assumptions  and  Limitations 

The  assumptions  made  in  this  thesis  are  as  follows.  It  was  assumed  the  SMEs 
were  aetually  subjeet  matter  experts  and  they  would  have  said  they  did  not  qualify  for  the 
study  if  that  was  the  ease.  This  assumption  was  validated  by  the  first  question  of  the 
interview  in  whieh  the  SMEs  were  asked  to  deseribe  their  experienee  with  rapid 
aequisition. 

Another  assumption  was  that,  eolleetively,  the  SMEs  interviewed  represent  a 
eross-seetion  of  the  rapid  aequisition  efforts  of  the  Air  Eorce.  Due  to  time  and 
availability  eonstraints  some  offiees  were  not  interviewed  or  were  unable  to  partieipate  in 
this  study.  As  sueh  the  generality  of  this  thesis  could  be  limited  by  the  lack  of  fully 
including  all  areas  of  Air  Eoree  rapid  aequisition. 

Summary 

This  seetion  discussed  the  methodology  of  the  interviews  and  data  analysis 
eondueted  for  this  thesis.  By  interviewing  SMEs  and  analyzing  their  eomments,  a  broad 
understanding  was  reaehed  regarding  what  the  Air  Eoree  does  in  support  of  rapid 
aequisition,  and  the  SE  proeesses  that  go  along  with  the  tailoring  that  is  done  to  ensure 
meeting  the  timeline.  These  results  are  discussed  in  Chapter  4. 
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IV,  Results 


Chapter  Overview 

The  purpose  of  this  chapter  is  to  examine  the  results  of  the  interviews  and  draw 
conclusions  from  the  data  to  answer  the  five  research  questions  posed  earlier. 

1 .  What  processes  does  the  United  States  Air  Force  use  to  complete  rapid  acquisition 
projects  and  programs? 

2.  Are  the  observed  processes  consistent  with  prescribed  instructions? 

3.  What  SE  activities  are  used  in  rapid  acquisition  programs  in  the  United  States  Air 
Force? 

4.  How  are  rapid  acquisition  programs  tailored  in  the  United  States  Air  Force? 

5.  Which  program  attributes  are  used  to  determine  program  tailoring? 

Each  interview  was  transcribed  verbatim  and  then  coded  and  categorized  based 
upon  the  content  provided  by  the  interviewees.  An  example  of  a  coded  portion  of  an 
interview  transcript  is  shown  below  in  Figure  7.  As  stated  earlier,  each  main  category 
meets  the  five  requirements:  be  responsive  to  the  research  questions,  be  as  sensitive  to  the 
data  as  possible,  be  exhaustive,  be  mutually  exclusive,  and  be  conceptually  congruent 
(Merriam,  2009). 
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Interview  #4  Completed  19  Dec  1300-1400 


[Interviewer]  Alright.  Now  with  your  experience  starting  from  the  Star  Wars  projects  until  now  what 
processes  have  you  seen  used  to  actually  complete  the  rapid  acquisition.  Here  I'm  looking  more  at  the 
acquisition  processes  versus  the  SE  activities.  Do  you  guys  have  Ols  that  you  follow?  Is  it  the  DAU 
acquisition  process,  but  you  guys  just  shrunk  it  down?  What  of  those  types  of  processes  have  you  guys 
seen? 


Ad-hoc 
Proces.s 
Small  Scope 


CP3 


[Respondent]:  It's  a  little  more  ad-hoc.  The  traditional,  the  DoD  5000  series,  acquisition  models,  j 
doesn't  really  work...but  it  works  for  large...or  more  appropriate  for  large  complex  systems.  What  we 
are  doing,  because  it  doesn't  have  to  have  all  of  the  auxiliary  DOTMLPF  functions,  because  it  really  is  a  } 
one  off  or  short  term  or  what  I  call  a  band  aid  type  problem  and  hoping  that  the  rest  of  the  acquisition 
system  will  catch  up.  So  it  has  been  streamlined  in  some  sense  and  has  really  been  relied  on  to  some  J 
extent  process  which  is  documented  in  AFRL's  instructions  views  of  the  activities  here  with  the 
innovation  team,  we  used  to  call  it  core  process  3,  CP3.  And  it's  fairly  loose,  it  involves  getting  the  right  J  Rqtitt  Dijf 
players  together  and  understanding  the  problem  and  the  core  root  of  the  problem  as  opposed  to  Stake¬ 

building  requirements  first  instead  of  documenting  the  needs  or  overaching  requirements.  It  really  is  a  }  Holder 
conversation  probing  of  the  operators  who  have  experienced  the  problems  and  root  cause,  developing  a  Comms 
OV-l  even  though  it  doesn't  involve  the  system  at  the  time,  and  how  it  is  that  a  solution  should  operate  |  System 
and  what  capabilities  it  should  provide  and  then  flowing  that  down  into  brainstorming  with  different  Design 
technologist,  various  solution  providers,  rapid  innovators,  what  things  could  be  brought  to  bear  to 
achieve  that  capability  or  circumvent  that  problem.  From  there  then  we  get  top  level,  we  have  a  1  Prgm  Start 
description  of  the  program  or  a  description  of  the  technical  solution  and  a  description  of  the  desired  Rqmts 

capabilities  or  requirements  with  a  little  r.  And  that  normally  starts  the  program,  and  as  the  program  is 
executed  those  evolve  in  close  collaboration  with  the  user.  So  we  identify  one  or  two  or  few  folks  who  1  Stake- 
represent  the  user  and  really  understand  what  the  issue  is  and  work  with  the  solution  team  to  adapt  the  /  Holder 
development  process  for  the  capabilities  and  solution,  or  the  quote  requirements.  As  we  run  into  Comms 

problems  that  we  can  do  with  technology  or  engineering  wise  and  other  things  that  they  see  as  useful 
capabilities  that  we  can  lash  on  during  that  process.  Sort  of  a  spiral  co-development  with  the  user,  and  ■> 
that's  been  effective  because  it  shortens  the  cycle  time  between  requirements  and  design  and  J 

generation,  but  it  does  require  the  rapid  prototyping  environment  to  build,  test  and  put  in  the  hands  of 
the  user.  To  ask  if  you  like  the  buttons  there,  to  you  like  this,  is  the  power  on  target  enough?  Is  it  too 
hard  to  fly?  Rather  than  formally  documenting  it's  more  of  this  ability  to  have  an  agile  development  1 
approach  with  a  core  team  that  has  the  capabilities  to  understands  what  the  user's  going  to  do  with  it 
and  how  it  can  be  done  from  an  engineering  point  of  view. 


Ad-Hoc 

process 


Figure  7  Example  Coding  of  Interview 

Figure  7  shows  an  example  of  the  first  stage  of  the  data  proeessing,  open  coding. 
This  is  followed  by  analytical  coding  where  the  open  codes  are  grouped  together.  In  the 
passage  shown  in  Figure  7  the  following  codes  were  grouped  together  based  upon  the 
content  that  they  represented;  Ad-hoc  Process,  CP3,  Spiral  Acquisition,  and  Ad-hoc 
Process.  When  this  group  was  combined  with  the  others  formed  during  the  open  and 
analytical  coding  of  the  ten  other  interviews  the  category  that  was  created  was  called 
Process.  A  full  listing  of  the  seven  categories  used  during  the  study  and  the  number  of 
codes  included  is  shown  in  Table  3.  8  shows  the  frequency  of  the  top  twenty-five  codes 
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used  during  this  study  and  helps  show  the  importanee  the  SMEs  plaeed  on  different 
themes.  A  full  listing  of  the  eodes  used  during  the  study  is  ineluded  in  Appendix  C. 

Table  3  Listing  of  Categories  and  Codes 


Category 

Data  Points 

Research  Questions 

Attributes 

52 

5 

Personnel  Experience 

12 

N/A 

Process 

116 

1,2 

Requirements 

52 

2,  3,  4,  5 

Solutions 

8 

3,4 

Systems  Engineering 

138 

3 

Tailoring 

67 

4 

Total 

445 

N/A 

Figure  8  Histogram  of  Code  Usage 


Each  category  was  associated  with  a  different  research  question  to  allow  for  the 


drawing  of  conclusions  as  shown  above  in  Table  3.  Each  category  is  not  tied  to  only  one 
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research  question;  for  example,  the  process  category  was  used  to  answer  research 
questions  1  and  2.  The  only  category  that  is  not  used  to  directly  answer  a  research 
question  is  Personnel  Experience;  this  was  used  to  categorize  the  experiences  of  each  of 
the  respondents  to  give  an  overall  narrative  of  the  personnel  interviewed.  This  category 
is  exempted  from  the  requirement  of  categories  being  responsive  to  the  study  as  it  ensures 
that  the  SMEs  meet  the  requirement  of  being  knowledge  experts  for  this  study. 

Question  1:  What  processes  does  the  United  States  Air  Force  use  to  complete  rapid 
acquisition  projects  and  programs? 

The  two  traditional  SPO  personnel  reported  using  the  QRC  process  to  answer  at 
least  one  rapid  acquisition  program  each,  while  the  lab  personnel,  as  a  majority,  did  not 
use  or  even  know  of  the  process.  One  lab  personnel  reported  having  previous  experience 
with  the  QRC  process. 

Another  process  that  was  reported  was  defined  in  AERLI  63-104.  All  seven  lab 
respondents  were  familiar  and  had  participated  in  that  process  to  conduct  rapid 
acquisition.  A  caveat  to  this  is  the  fact  that  an  ad-hoc  process  was  reported  to  be  used  by 
all  seven  of  the  respondents. 

Both  of  the  traditional  SPO  respondents,  all  of  the  rapid  SPO  respondents  and  five 
of  the  lab  respondents  reported  that  the  acquisition  process  that  they  currently  use  to 
answer  rapid  acquisition  requests  were  currently  ad-hoc  processes  that  are  ill-defined  in 
instructions  or  literature.  That  was  not  to  say  that  they  were  undisciplined,  as  two  of  the 
respondents  showed  that  they  had  a  process  that  their  organization  used  for  successfully 
completing  over  170  rapid  projects.  One  of  the  traditional  SPO  respondents  described 
how  their  process  was  based  on  API  63-114;  however  they  were  not  conducting  a 
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designated  QRC  and  hence  were  not  considered  for  that  process.  One  of  the  rapid  SPO 
respondents  discussed  how  their  process  was  managed  predominately  by  personality  and 
varied  greatly  between  each  PM  and  program.  The  reported  results  can  be  seen  in  Figure 

9. 


Processes  used  (not  mutually  exclusive) 


Figure  9  Reported  Process  Utilization 

This  result  differs  from  the  perception  of  a  standard  process  used  throughout  the 
USAF  to  complete  rapid  acquisition.  The  interviewees  likened  this  to  the  difference  in 
their  perspective  of  the  work  to  be  completed.  The  lab  personnel  looked  at  the  rapid 
acquisition  process  as  trying  to  accomplish  smaller  scope  programs  with  limited 
quantities  of  an  item  being  produced.  The  laboratory  programs  were  normally  started  by 
indirect  discussions  with  the  COCOMs  and  not  a  capability  gap  declared  in  a  UON  or 
JUON.  The  traditional  SPOs  looked  at  larger  solutions  that  were  down  scoped  to  allow 
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for  a  faster  program  but  produce  a  larger  amount  of  items  due  to  the  need  to  implement 
across  a  fleet  of  aircraft. 

Question  2:  Are  the  observed  processes  consistent  with  prescribed  instructions? 

The  data  show  there  are  multiple  processes  that  the  USAF  uses  to  complete  rapid 
acquisition.  When  examining  if  the  QRC  process  defined  in  API  63-114  was  used  to 
answer  JUONs  and  UONs  from  the  warfighter  it  was  reported  by  the  respondents  that  the 
majority  of  them  did  not  use  the  QRC  process.  This  can  be  seen  in  Figure  10. 


Figure  10  Reported  Usage  of  the  QRC  Process 
This  question  examined  whether  the  processes  that  were  observed  in  the  first 
research  question  followed  prescribed  instructions.  These  instructions  could  be  AFI  63- 
1 14,  AFRLI  61-104  or  any  other  defined  instruction  that  was  approved  by  the  USAF. 
When  examining  the  different  responses  the  answer  becomes,  as  one  respondent 
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mentioned,  the  classieal  PM  answer  ...  ‘it  depends.’  One  rapid  SPO  respondent  stated 
that  they  do  not  normally  work  QRC  programs  due  to  political  issues  and  infighting  from 
the  stakeholders  of  the  UON/JUON.  They  still  answer  UONs  and  JUONs,  but  they  are 
not  designated  as  QRCs  and  therefore  do  not  fall  under  API  63-1 14. 

As  shown  previously  in  Figure  lOError!  Reference  source  not  found.,  the  traditional 
SPOs  do  follow  the  process  as  defined  in  API  63-1 14  for  QRC  designated  programs. 
However,  when  working  on  a  rapid  acquisition  program  that  was  not  designated  as  a 
QRC,  the  traditional  SPOs  used  an  ad-hoc  process  that  was  based  on  the  QRC  process  but 
more  tailored  to  the  requirements  for  that  program.  As  stated  by  the  PM  “we  didn’t  have 
a  [. . .]  QRC  that  was  initiated  with  a  63-1 14  but  we  said  this  is  something  we  can  do  in  a 
rapid  process  or  a  rapid  manner  and  meet  a  fairly  aggressive  schedule.” 

While  the  lab  personnel  reported  that  they  work  in  an  ad-hoc  manner,  they  do 
base  their  decisions  on  the  processes  as  prescribed  in  AFRLI  61-104.  They  still  do  not 
follow  the  process  strictly;  however  they  use  that  as  a  starting  point  and  then  tailor  from 
there. 

As  mentioned  previously  in  Figure  9,  one  of  the  two  rapid  SPOs  does  not  follow 
any  prescribed  process,  while  the  other  organization  does  not  follow  API  63-1 14  as  the 
programs  they  work  on  are  not  UONs  or  JUONs.  This  organization  has  its  own  process 
due  to  the  fact  they  are  not  tasked  with  completing  the  full  program;  their  programs  are 
based  upon  taking  the  need  of  the  user  and  designing  a  working  solution  in  less  than  six 
months.  This  solution  is  then  passed  to  a  program  office  to  be  included  in  a  more  formal 
acquisition  program,  either  as  a  legacy  update  or  rapid  acquisition  program. 


41 


Question  3:  What  SE  activities  are  used  in  rapid  acquisition  programs  in  the  United 
States  Air  Force? 

When  examining  what  SE  aetivities  were  used,  the  sixteen  SE  aetivities  ealled  out 
in  the  DAG  were  the  starting  point.  Eaeh  respondent  was  given  the  list  shown  in 
Appendix  B  and  was  asked  if  their  organization  used  those  aetivities.  As  a  general 
answer,  most  of  the  respondents  stated  they  do  ineorporate  Systems  Engineering  into 
their  programs.  However,  the  delineation  here  eomes  in  the  form  that  not  all  programs 
use  the  same  level  of  SE  rigor  and  not  all  SE  aetivities  are  eondueted.  The  aggregated 
results  are  shown  later  in  Eigure  1 1  and  Eigure  12,  and  are  tabulated  as  a  pereentage  for 
the  respondents  who  answered  that  they  do  ineorporate  the  SE  aetivity  in  their  programs. 
General  Findings 

Eor  the  traditional  SPO  personnel,  it  was  reported  that  SE  was  important  to  their 
proeesses.  They  attempted  to  use  diseipline  and  rigor  when  working  through  their  rapid 
aequisition  proeess.  The  seope  of  the  problem  being  addressed  played  a  large  part  in 
deeiding  how  mueh  of  the  SE  aetivities  to  inelude  in  the  programs.  There  were  reported 
instanees  of  the  SE  aetivities  being  iterative,  however  it  was  noted  that  those  iterations 
were  minimized  to  expedite  the  proeess. 

The  lab  viewed  SE  through  the  lens  of  AERLI  61-104,  with  the  eight  questions 
taking  the  plaee  of  the  elassieal  SE  nomenelature.  The  effort  foeused  primarily  on  the 
Teehnieal  Proeesses,  with  an  iterative  approaeh  being  used. 

The  rapid  SPO  personnel  diseussed  how  they  eomplete  the  SE  aetivities,  but  do 
not  use  the  elassieal  nomenelature  due  to  emotional  responses  to  it.  As  one  respondent 
stated  “you  mention  a  word  and  it  invokes  in  the  person  what  they  think  [. . .]  needs  to  be 
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done.”  The  respondents  stated  that  they  focused  on  the  intent  and  completed  most  of  the 
SE  activities  as  a  whole  instead  of  stating  the  completion  of  each  individual  activity. 

Findings  on  the  Technical  Processes 


Figure  1 1  Reported  Technical  Processes  Used 


When  examining  the  responses  with  respect  to  the  first  Technical  Process, 
Stakeholder  Requirements  Development,  the  establishment  of  communications  with  the 
stakeholder  seemed  to  be  the  biggest  concern  with  almost  half  of  the  participants  (5  of 
12)  stating  such.  There  were  minimum  conversations  about  other  activities  called  for 
during  this  process.  It  should  be  noted  that  in  Figure  1 1  and  the  following  figures,  due  to 
the  scarcity  of  the  SMEs  in  the  organizational  categories,  some  categories  have  an 
artificial  step  function.  This  can  be  seen  in  the  traditional  SPO  response  above;  if  both 
respondents  answer  yes,  then  it’s  100%,  while  one  respondent  gives  an  answer  of  50%. 
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Requirements  Analysis  was  utilized  less  by  the  traditional  SPO  personnel  with  a 
reported  positive  eorrelation  to  seope  in  that  as  the  programs  seope  beeame  larger,  the 
amount  of  time  spent  performing  requirements  analyses’  grew.  The  lab  personnel 
reported  they  ineluded  the  requirements  analysis  aetivity  at  a  higher,  more  eoneeptual 
level  “but  enough  to  kind  of  provide  the  top  level  vision  of  where  we  want  to  go  as  an 
organization.  As  we  get  down  to  the  speeifie  programs,  requirements  management,  there 
is  a  proeess  for  that.  I’d  say  that  proeess  is  reasonable,  but  again,  in  an  attempt  to  get  to 
other  parts  of  the  program,  sometimes  it  gets  watered  down  a  bit.” 

Architeeture  Design  was  reported  as  low  aeross  all  of  the  respondents,  with  one  of 
the  traditional  SPO  respondents  diseussing  how  they  will  allow  the  eontraetor  for  the 
aireraft  to  address  it  due  to  the  small  seope  of  their  programs.  Inside  the  lab,  only  two  of 
the  personnel  diseussed  working  arehiteeture  aetivities,  with  one  stating  it  is  something 
that  their  organization  is  attempting  to  improve. 

With  Implementation,  only  one  traditional  SPO  and  lab  personnel  eaeh  diseussed 
it,  with  both  stating  that  they  eomplete  this  phase  during  their  programs.  An  assumption 
may  be  made  stating  that  to  eomplete  a  program  you  must  eompete  the  aetivities  that  are 
diseussed  in  this  proeess,  therefore  the  respondents  assumed  that  it  was  eompleted  and 
did  not  mention  it  during  the  interviews. 

With  Integration,  one  of  the  traditional  SPO  personnel  stated  that  “that’s  about  all 
we  do.  We  take  things  that  are  already  developed  and  put  them  on  the  airplane.”  The  lab 
personnel  stated  that  this  step  worked  in  eoneert  with  the  Verifieation  proeess  during 
their  projeets,  where  the  Verifieation  proeess  would  show  them  areas  that  they  did  not 
integrate  eorreetly  and  would  require  more  design  work. 
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The  next  two  proeesses,  Verifieation  and  Validation,  work  hand  in  hand.  The 
traditional  SPO  discussed  conducting  both  the  developmental  test  (DT)  and  the 
operational  test  (OT)  at  the  same  time  to  accelerate  the  process  by  reducing  the  number 
of  tests  needed.  The  lab  respondents  stated  that  less  than  half  complete  the  verification 
and  validation  activities.  This  is  not  to  say  that  they  don’t  test  their  products  or  designs, 
but  that  they  do  not  consider  what  they  do  to  be  part  of  these  two  SE  activities. 

The  last  TP  to  discuss  is  the  Transition  process.  One  of  the  traditional  SPO 
personnel  discusses  the  questions  that  they  face  after  fielding  their  programs.  “Now  that 
the  thing  is  out  there  what  are  we  going  to  do  with  it?  Is  it  programmed  for?  Is  it  spared 
for?  We  don’t  normally  get  into  big  logistical  source  and  repair  analysis,  normally  it’s  ten 
percent  spares  for  QRCs.  It’s  kind  of  the  rule  of  thumb.”  This  contrasts  with  the  lab, 
where  they  recognize  a  “valley  of  death”  between  the  lab  programs  and  transitioning  the 
programs  to  the  SPOs. 

One  of  the  respondents  from  the  Rapid  SPOs  discussed  their  SE  processes  used, 
which  focused  mainly  on  the  different  Technical  Processes  described  in  the  DAG  (DAG). 
“So  you’ve  got  your  customer,  you  identify  stakeholders;  you  go  through  determining 
your  requirement  analysis.  Can  we  do  it?  Eet’s  hold  a  meeting  with  the  customer.  Now 
mind  you,  this  might  be  [all  during]  day  one.  [Starting  of  the  design  process]  might  be 
day  five,  and  we  deliver  at  day  30.  That’s  how  fast  it  is.” 
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Findings  on  the  Technical  Management  Processes 


Figure  12  Reported  Technical  Management  Process  Used 


As  seen  in  Figure  12,  the  first  two  TMPs,  Decision  Analysis  and  Technical 
Planning  were  not  described  as  being  used  during  the  rapid  acquisition  programs 
discussed  during  the  interviews.  The  third  TMP,  Technical  Assessment  was  how  the 
respondents  examined  the  technical  aspects  of  their  programs.  One  of  the  traditional  SPO 
personnel  discussed  how  their  reviews  were  at  two  levels,  first  was  the  weekly 
assessments  of  the  program  run  by  the  PM  and  the  second  was  the  more  formal 
Preliminary  Design  Review  (PDR)  and  Critical  Design  Review  (CDR)  conducted  by  the 
PEO.  The  lab  personnel  reported  completing  technical  reviews,  such  as  PDRs  and  CDRs 
or  semi-annual  program  reviews.  The  respondents  from  the  Rapid  SPOs  discussed 
competing  “design  develop,  design  approved,  design  released”  reviews  and  PDRs  and 
CDRs  that  are  much  closer  to  each  other,  in  the  range  of  2-3  months  apart. 
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The  fourth  TMP  is  Requirements  Management,  and  only  had  one  response  from  a 
Lab  personnel  stating  that  “they  did  determine  roles  and  responsibilities”  but  that  the 
other  aetivities  listed  for  this  TMP  were  not  conducted.  This  might  be  from  the 
respondents  lumping  activities  that  could  be  considered  Requirements  Management  into 
one  of  the  other  TPs  such  as  Stakeholder’s  Requirements  Development  or  Requirements 
Analysis. 

Risk  Management  was  discussed  by  both  of  the  traditional  SPO  personnel  and 
included  the  risk  planning,  identification  and  analysis  sub-activities.  Both  respondents 
discussed  how  Risk  Management  is  used  in  planning  and  executing  their  programs.  The 
lab  personnel  stated  that  Risk  Management  varies  between  programs  with  two  of  the  six 
stating  that  they  do  not  use  it  while  three  other’s  state  that  they  do  include  this  activity  in 
their  programs.  One  stated  its  importance  as  “when  you’re  on  a  fast  moving  train  and 
things  starts  falling  off  or  things  start  rattling  around  and  shaking  you  need  to  have 
already  thought  three  steps  ahead  and  be  thinking  about  all  the  possible  contingencies  so 
you’ve  got  plan  b  and  plan  c  so  you  can  quickly  implement  so  you  don’t  have  a  train 
wreck,  or  stop  the  train.” 

Configuration  Management,  from  the  traditional  SPO  personnel’s  perspective, 
grows  with  an  increase  in  the  size  of  the  programs  being  managed.  Ensuring  software 
and  hardware  configurations  are  the  same  throughout  the  program  and  the  fleet  of 
equipment  being  serviced  is  important.  From  the  lab’s  perspective,  this  TMP  is  used, 
however  with  varying  degrees  of  rigor.  Two  respondents  stated  that  what  Configuration 
Management  is  completed  is  very  minimal,  while  three  stated  that  it’s  completed,  but  it 
tends  to  be  corrected  closer  to  the  transition  of  the  program  and  it’s  fielding. 
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Technical  Data  Management  was  discussed  by  one  of  the  traditional  SPO 
respondents  in  response  to  issues  that  they  had  perceived  with  the  contractor  that  they  use 
and  the  use  of  Independent  Research  and  Development  (IRAD)  money  previous  to  the 
program.  The  lab  respondents  stated  that  this  TMP  was  poorly  applied  across  the 
different  programs  and  leadership  had  begun  to  try  and  drive  more  discipline  into  the 
programs.  One  unique  item  that  was  discussed  came  from  the  rapid  SPOs,  due  to  the  fact 
that  as  the  rapid  design  and  manufacturing  capability  inherent  in  the  organization  they  are 
the  provider  of  technical  data  to  many  programs  that  they  work  with.  After  they  have 
completed  the  design  they  help  those  programs  leverage  that  data  into  rapidly  procuring 
larger  numbers  of  items  from  contractors. 

The  final  TMP  is  Interface  Management.  The  traditional  SPO  personnel 
conducted  Interface  Management  due  to  the  systems  that  they  are  in  charge  of.  Their 
viewpoint  of  it  is  “here  is  the  interface  for  the  airplane,  build  your  widget  to  that.”  The 
lab  personnel  discussed  the  TMP,  however  less  than  half  indicated  that  it  was  an  activity 
that  they  include  with  their  programs. 

Findings  on  the  Initiation  of  Programs 

There  were  four  main  initiation  points  discussed  during  the  interviews  as  seen 
below  in  Figure  13.  Three  were  listed  in  the  literature  review:  a  JUON  or  UON  from  the 
warfighter,  AF/CC  request  and  designation,  and  technology  push.  AFI  63-114  discussed 
the  first  two,  and  AFRLI  63-104  was  the  instruction  that  discussed  how  technology  from 
the  lab  can  initiate  a  rapid  acquisition  program.  The  other  option  that  was  discussed  by 
the  SMEs  was  that  of  informal  requirements  initiation. 
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The  informal  method  was  almost  exclusively  used  in  the  laboratory,  with  it  being 
described  as  “direct  conversations  with  leaders  of  COCOMs  or  MAJCOMs  and  asking 
the  two  star  or  three  star  ‘what  is  it  that  is  really  bothering  you  right  now?  If  you  had  a 
solution  within  in  a  year,  what  can  the  Lab  do  for  you?  What  is  the  most  pressing  need  to 
be  solved?’”  This  was  only  slightly  different  for  the  one  respondent  from  the  rapid  SPOs 
who  stated  that  they  received  some  of  their  program  initiations  from  organizations  based 
upon  phone  calls  and  emails  from  PMs  whom  they’ve  briefed  on  their  abilities. 


Figure  13  Reported  Program  Initiation 

There  was  another  unique  item  brought  up  by  the  respondents  associated  with 
UONs  and  JUONs,  in  that  many  times  the  respondents  discussed  that  they  would  have 
JUONs  written  for  specific  material  needs  or  solutions  that  are  already  available.  This  is 


different  then  what  the  literature  states  should  happen  where  the  UON  or  JUON  states  the 
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capability  gap.  As  one  respondent  stated  “Usually  the  user  or  sponsoring  organization 
(SAF/AQI,  JIEDDO,  etc)  sees  the  teeh  (aka  shiny  widget)  in  a  briefing  or  demonstration 
and  decides  they  need  it  right  away.  The  requirement  is  then  erafted  as  an  ‘I  need  that 
shiny  widget’  UON/JUON.” 

One  respondent  from  both  the  traditional  SPO  and  rapid  SPO  stated  that  they 
began  programs  based  upon  an  AF/CC  initiation.  These  were  stated  to  be  high  priority 
programs  that  had  oversight  and  priority  based  upon  the  Chief  of  Staff  of  the  Air  Foree 
interest  in  the  programs. 

Findings  on  Systems  Engineering  Iterations 

There  was  a  stark  difference  in  the  viewpoints  of  how  iterative  the  SE  processes 
were  between  the  personnel  groups.  The  traditional  SPO  personnel  discussed  how  they 
believe  that  the  proeesses  are  iterative;  however  they  have  to  try  to  minimize  the  amount 
of  iterations  to  accelerate  the  program  and  due  to  management  believing  that  iterations 
mean  that  you  were  unable  to  eorrectly  eomplete  the  proeess  the  first  time  and  requires 
rework. 

The  Eab  personnel  believe  that  the  proeesses  are  very  iterative.  The  belief  held 
by  lab  personnel  was  that  you  would  stop  the  SE  activities  when  the  team,  to  include  the 
PM  and  Systems  Engineer,  was  satisfied. 

Question  4:  How  are  rapid  acquisition  programs  tailored  in  the  United  States  Air 
Force? 

Tailoring  was  conducted  mostly  in  a  bottom-up  methodology  where  the  PM  or 
Systems  Engineer  started  with  some  type  of  minimum  baseline  that  included  processes 
that  they  believe  all  programs  must  complete  to  be  successful.  SE  and  acquisition 
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activities  and  processes  were  added  from  that  point  to  build  up  to  the  final  program.  One 
respondent  discussed  how  “we  had  a  deadline  to  meet  and  we  wanted  to  follow  a 
disciplined  approach  to  the  process  so  we  took  out  things  that  didn’t  add  value  but  didn’t 
create  undue  risk  for  the  project. 

Another  viewpoint  that  was  brought  up  during  the  interviews  was  the  concept  of 
continually  tailoring  the  process  from  beginning  to  end  depending  on  how  the  program 
was  going  and  the  required  work  necessary  to  complete  the  program.  As  stated  by 
another  respondent,  “Then  as  we  start  executing  we  peel  back  the  onion  a  little  bit.  We 
get  a  better  understanding  of  the  underlying  interfaces  and  the  problems  with  those 
interfaces  and  other  critical  parameters  that  need  to  be  monitored  and  tracked, 
particularly  now  that  we  understand  the  relationship  between  the  underlying  parts.  So 
generally  the  controls  get  added  as  we  move  along.”  This  is  contrasted  against  the  need 
to  understand  how  much  tailoring  to  do.  One  reported  way  to  decide  how  much  tailoring 
to  do  was  using  the  expected  level  of  effort  for  the  program,  with  an  inverse  correlation. 
So  as  less  effort  is  being  put  forth  for  the  program,  you  would  expect  more  tailoring  of 
activities,  and  vice-versa.  Another  respondent  stated  “when  [tailoring  activities]  starts 
hindering  the  execution  you’ve  gone  too  far.” 

The  most  important  aspect  for  tailoring  seemed  to  be  the  use  of  the  experience  of 
the  personnel  on  the  team  along  with  the  PM  and  Systems  Engineer.  All  of  the 
respondents  mentioned  that  they  choose  which  activities  to  include  based  upon  the 
recommendations  of  their  personnel  or  their  own  experiences.  No  one  mentioned  any 
guidance  given  in  the  APIs  or  elsewhere  in  determining  what  processes  must  be 
completed  or  to  what  level.  As  one  respondent  mentioned  “the  trick  comes  down  to 
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having  the  right  person  with  the  right  knowledge  making  the  deeision  based  upon  their 
experienee,  their  baekground  and  their  understanding  of  the  problem.” 


Question  5:  Which  program  attributes  are  used  to  determine  program  tailoring? 

There  were  multiple  attributes  that  were  provided  for  determining  how  to  tailor 
the  programs:  money,  technology  maturity,  manufacturing  readiness,  risk,  scope, 
requirements  definition,  schedule,  systems  integration,  and  acceptable  levels  of  success 
and  can  be  seen  below  in  Figure  14. 


■  TiacUtional  SPO 
□  AFRL 

■  Rapid  SPO 


Figure  14  Reported  Attributes  Used  for  Tailoring 
Many  of  the  attributes  are  easily  explained;  a  high  level  of  risk  would  require 
more  effort  to  mitigate  the  chances  of  failure,  while  low  money  levels  might  mean  a 
smaller  level  of  effort  or  available  resources.  Technology  maturity  determines  how  much 
effort  must  be  spent  in  preparing  different  ideas  before  they  are  usable  in  the  system.  For 
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example,  a  high  teehnology  maturity  ean  be  seen  as  a  commereial  off  the  shelf  (COTS) 
solution  ready  for  integration  into  a  different  environment. 

Requirements  definitions  was  an  outreaeh  from  the  SE  proeess  of  the  same  name. 
Multiple  respondents  stated  that  “requirements  ereep”,  as  defined  as  a  ehange  or  inerease 
in  the  system  requirements,  has  a  negative  impact  on  the  success  of  a  program.  One 
respondent  stated  that  by  “bin  and  freezing”  their  requirements,  the  program  was  able  to 
minimize  requirements  creep  and  turn  out  the  first  iteration  of  the  program  on  schedule 
with  a  second  program  to  meet  future  needs. 

The  acceptable  level  of  success  for  a  program  ties  into  requirements  definition, 
but  can  stand  alone  as  an  attribute.  Multiple  respondents  stated  that  their  customers 
would  be  thrilled  with  an  80%  solution  now,  with  one  stating  that  “sometimes  we  provide 
a  50%  solution  and  that’s  acceptable.” 
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V.  Conclusions 


The  standard  acquisition  process  is  unable  to  meet  the  urgent  needs  of  warfighters 
in  today’s  combat  operations.  Knowing  this,  the  DoD  as  a  whole,  and  the  Air  Force  as  a 
Service,  have  been  executing  rapid  acquisition  programs.  This  study  set  out  to  identify 
from  PMs  and  Systems  Engineers  which  processes  they  used  to  conduct  these  programs 
and  how  they  tailored  them  to  meet  the  expedited  timelines.  It  was  conducted  as  a 
qualitative  study,  using  semi-structured  interviews  to  gather  data  from  the  SMEs, 
followed  by  coding  and  cataloging  the  responses. 

There  are  many  processes  used  throughout  the  Air  Eorce  to  conduct  rapid 
acquisitions.  The  QRC  process  defined  in  AEI  63-114  along  with  the  AERL  process 
defined  in  AERLI  63-104  were  discussed  by  respondents.  One  organization  that 
conducted  rapid  design  and  production  discussed  their  process  as  being  decidedly 
different  than  either  of  the  previously  mentioned  processes  and  redesigned  based  upon 
sound  engineering  principles.  Another  respondent  described  their  process  as  being  a 
personality  driven  ad-hoc  process  that  followed  the  experience  of  the  PM  and  Systems 
Engineer.  Including  this  respondent,  9  of  the  12  respondents  described  experience 
working  projects  using  an  ad-hoc  methodology  for  the  acquisition  and  design  process. 
These  ad-hoc  processes  were  used  in  the  majority  of  the  programs  that  were  discussed 
and  by  definition  did  not  follow  any  prescribed  process  methodology.  This  is  not  to  say 
that  they  were  following  a  less  rigorous  process  or  failing  to  meet  the  warfighter’s 
requirements  and  needs  in  a  timely  fashion.  It  does  support  the  assertion  found  in  the 
GAO  reports  stating  that  there  is  a  lack  of  standardization  in  how  rapid  acquisition  is 
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conducted.  However,  many  of  the  respondents  believe  that  the  bureaueraey  that  eomes 
with  standardization  is  one  of  reasons  that  programs  take  longer  than  the  time  required 
for  rapid  aequisition. 

When  examining  the  different  SE  proeesses  the  majority  of  the  programs  reported 
using  the  different  TPs  aeross  all  three  organizational  groups.  Looking  at  the  TMPs  it 
ean  be  seen  that  Deeision  Analysis,  Technieal  Planning  and  Requirements  Management 
were  not  discussed  by  the  respondents  in  any  appreeiable  way.  This  does  not  mean  that 
these  proeesses  were  not  used  by  any  of  the  SMEs,  but  it  does  show  that  they  are  less 
valued  when  compared  to  the  other  TMPs.  Here  the  laek  of  a  eommon  language  and 
standardized  proeesses  make  eomparing  the  proeesses  diffieult,  more  so  when 
eompounded  by  the  faet  that  multiple  organizations  were  ineluded  in  the  study. 

It  was  noted  that  all  partieipants  did  state  that  SE  activities  are  iterative  but  there 
is  a  difference  between  the  traditional  SPO  and  lab  personnel  when  diseussing  for  how 
long.  The  traditional  SPO  personnel  stated  that  they  try  to  minimize  the  number  of 
iterations  to  aoeelerate  the  proeess,  while  the  Lab  personnel  stated  they  iterate  until  they 
feel  that  they’ve  met  the  need.  This  eould  be  explained  by  the  differenees  in  the  projeets. 
The  traditional  SPO  programs  are  normally  using  mature  teehnology  and  only  need  to 
integrate  them  into  the  overall  system.  As  one  respondent  stated,  “one  of  the  things  that 
lent  [the  program]  to  being  a  rapid  aequisition  is  the  faet  that  it  is  what  I  would  eonsider 
to  be  a  federated  system.”  This  ean  be  different  from  the  AERL  programs,  whieh  can  be 
defined  as  Joint  Rapid  Aequisition  Cell  programs  where  the  time  line  is  “fewer  than  two 
years  with  little  or  no  development”  (AERL,  2013a). 
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When  looking  at  how  the  programs  were  tailored,  10  of  the  12  respondents 
eonsidered  the  tailoring  to  be  bottom-up.  This  methodology  ean  be  defined  as  some 
standard  baseline  that  all  of  the  organizations  programs  had  to  meet  with  aetivities  added 
based  on  the  program  oharaeteristies  and  inputs  of  the  Systems  Engineer  and  PM.  All  of 
the  respondents  diseussed  that  deeisions  on  how  to  tailor  were  based  on  the  experienee  of 
the  PM  and  Systems  Engineer.  Another  point  for  tailoring  that  was  made  was  the  need  to 
ensure  that  the  customer  will  accept  a  less  than  optimal  solution.  As  stated  by  one 
respondent  “perfect  is  the  enemy  of  good  enough.”  This  could  be  an  instance  of  the  so 
called  Pareto  principle,  where  an  80%  solution  would  suffice,  but  the  program  would 
break  the  time  suspense  if  it  attempted  to  design  a  full  100%  solution. 

Examining  requirements  generation  the  instructions  state  that  requirements  should 
be  generated  by  a  LION  or  JEION,  direction  from  the  AE/CC,  or  the  push  of  new 
technology  from  the  laboratory,  this  research  found  that  a  fourth  means  of  generation  was 
used.  The  informal  generation,  while  not  harmful  to  the  programs  can  allow  programs  to 
begin  and  use  material  and  funding  that  might  not  meet  requirements  that  are  defined  by 
the  JCIDS  process  or  top  leadership. 

Of  all  the  program  attributes  discussed  by  the  respondents,  risk,  technology 
maturity  and  a  lack  of  requirements  creep  were  the  three  most  discussed  attributes  when 
examining  programs  to  decide  how  to  tailor.  A  high  level  of  either  technical  or 
managerial  risk  would  be  difficult  to  plan  for,  while  a  high  technology  maturity  would 
ensure  that  the  program  could  focus  on  integration  of  the  technology  into  the  system 
versus  maturing  the  technology  for  use.  Requirements  creep  can  derail  programs  by 
continually  changing  the  target  for  the  program.  One  traditional  SPO  respondent 
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discussed  how  they  were  able  to  “bin  and  freeze”  the  requirements,  whieh  allowed  a 
focused  effort  with  an  understanding  that  other  issues  that  were  brought  up  by 
stakeholders  would  be  dealt  with  in  a  future  iteration  of  the  program. 

Limitations 

As  noted  in  ehapter  one,  the  access  to  personnel  limited  the  inelusion  of  all 
personnel  in  this  study.  Some  of  the  aequisition  personnel  work  on  elassified  programs 
that  preelude  them  from  participating  in  the  study  while  others  were  unable  or  unwilling 
to  partieipate. 

Another  aecess  issue  was  the  use  of  the  snowball  sampling,  which  was  able  to 
inerease  the  total  number  of  partieipants.  However,  it  does  not  guarantee  that  you  are 
sampling  from  all  of  the  available  personnel.  This  laek  of  a  fully  developed  sample  of 
SMEs  diminishes  the  ability  of  the  results  to  be  fully  transferrable  to  all  Air  Foree  rapid 
aequisition  programs.  A  full  set  of  SMEs  should  inelude  all  offiees  that  eonduct  rapid 
aequisition  and  inelude  the  spaee  domain,  aireraft,  eyberspaee,  laboratory,  and  speeial 
operations. 

The  laek  of  the  researeher’s  familiarity  with  interviewing  allowed  the  respondents 
to  wander  off  track  from  the  questions  and  also  to  eontrol  the  information  flow.  This 
limitation  eould  be  the  reason  why  some  of  the  partieipant’s  information  was  elearly 
defined  and  others  were  vague  and  eould  negatively  impact  the  internal  validity  of  the 
results. 
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Recommendations  for  Future  Research 


Future  research  could  be  completed  by  reducing  the  scope  of  the  study  and 
focusing  solely  on  the  interactions  between  the  different  SE  processes.  By  reducing  the 
scope,  the  researcher  can  focus  the  study  and  provide  a  deeper  analysis  of  the  interactions 
between  the  processes  to  allow  for  PM  and  System  Engineers  to  fully  understand  the 
results  of  how  minimizing  one  activity  impacts  the  other  activities  and  the  impact  on 
successfully  answering  a  rapid  acquisition  program. 

Another  future  area  of  research  would  include  the  building  of  multiple  case 
studies  of  different  rapid  acquisition  programs  to  allow  for  an  in-depth  look  into  how  they 
are  executed.  These  case  studies  can  be  compared  against  each  other  to  create  best 
practices  that  are  rooted  in  multiple  programs. 

Conclusions 

This  study  resulted  in  four  major  conclusions.  The  first  is  that  there  are  many 
processes  used  in  the  Air  Eorce  for  executing  rapid  acquisition  programs  and  most  of 
them  do  not  use  prescribed  processes.  This  can  imply  that  the  acquisition  corps  as  a 
whole  does  not  follow  a  standardized  set  of  best  practices  nor  is  there  a  corporate 
memory  for  those  best  practices.  While  the  Air  Eorce  attempted  to  use  the  QRC  process 
as  a  way  of  meeting  urgent  needs  of  the  warfighters,  it  seems  that  the  expected 
bureaucracy  of  the  QRC  process  has  made  senior  leadership  allow  programs  to  be  run 
without  following  prescribed  processes.  This  does  not  mean  that  due  diligence  or  proper 
Program  Management  concepts  were  not  applied,  but  it  does  indicate  that  personnel 
would  prefer  a  less  cumbersome  process. 
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Another  finding  was  that  while  most  of  the  respondents  indieated  they  used  the 
TPs,  not  all  of  the  TMPs  were  used.  Many  of  the  programs  used  only  some  of  the 
Technical  Management  Processes  even  though  they  understood  that  it  could  cause 
problems.  This  indicates  that  either  the  processes  don’t  provide  the  expected  benefits  to 
rapid  acquisition  programs  or  that  the  time  and  effort  requirement  for  those  processes 
could  be  too  high  for  the  programs  to  use. 

The  third  major  finding  was  that  no  respondent  stated  that  they  used  any  outside 
guidance  when  deciding  how  to  tailor  their  programs  other  than  the  experience  of 
themselves  or  their  team.  This  could  indicate  a  lack  of  dissemination  of  information  from 
previous  research,  a  lack  of  use  of  best  practices  as  seen  in  the  first  finding,  or  a  missed 
opportunity  for  education  through  the  DAU.  This  lack  of  information  passage  could  be 
the  underpinnings  of  other  issues  in  the  acquisition  corps.  A  recommendation  to  address 
this  problem  is  to  create  a  center  of  excellence  for  rapid  acquisition  as  a  sub-center 
underneath  either  the  Acquisition  Center  of  Excellence  (ACE)  or  DAU.  This  center’s 
focus  would  be  the  collection  of  lessons  learned  and  the  dissemination  of  the  basic 
knowledge  to  the  members  who  are  conducting  rapid  acquisition.  This  can  be  in  concert 
with  DAU  classes  that  are  required  coursework  for  members  of  the  acquisition 
profession. 

The  final  major  finding  is  that  a  lack  of  requirements  creep,  risk  and  technology 
maturity  are  attributes  programs  look  at  when  deciding  to  tailor.  These  three  are 
attributes  that  were  included  in  other  research  studies;  however,  these  were  the  only  ones 
that  resonated  with  the  interviewees  in  the  study. 


59 


References 


Air  Force  Research  Laboratory.  (2011)  Laboratory  Personnel  Demonstration  Project. 
AFRLM  36-104.  Wright-Patterson  AFB 

Air  Force  Research  Laboratory.  (2013a).  Scientific/Research  and  Development.  AFRLl 
61-108.  Wright-Patterson  AFB 

Air  Force  Research  Laboratory.  (2013b).  Science  And  Technology  Systems  Engineering 
and  Technical  Management.  AFRLl  61-104  Wright-Patterson  AFB 

Beasley,  R.,  &  Partridge,  R.  (2010).  “The  Three  Ts  of  Systems  Engineering  -  Trading 
Tailoring,  and  Thinking,”  20th  INCOSE  International  Symposium  in  Chicago, 

Behm,  S.  M.,  Pitzer,  J.  B.,  &  White,  J.  F.  (2009).  A  Tailored  Systems  Engineering 
Framework  for  Science  and  Technology  Projects.  (No.  AFIT/GSE/ENV/09-M02) . 
AIR  FORCE  INSTITUTE  OE  TECHNOEOGY  WRIGHT-PATTERSON  AFB  OH 
SCHOOE  OF  ENGINEERING  AND  MANAGEMENT. 

Bui,  Y.  N.  (2014).  In  Knight  V.,  et.  al.  (Eds.),  How  To  Write  A  Master's  Thesis  (2nd  ed.). 
Eondon,  United  Kingdom;  SAGE  Publications,  Incorporated. 

Defense  Acquisition  University.  (2004).  Defense  Acquisition  Guidebook.  Alexandria,  Va: 
DAU. 

Defense  Acquisition  University.  (2014).  Integrated  Defense  Acquisition,  Technology,  and 
Logistics  Life  Cycle  Management  System.  Retrieved  7  Feb,  2014,  from 
https://ilc.dau.mil/ 

Department  of  the  Air  Force.  (2008).  Fact  sheet:  F-22  raptor.  Retrieved  February  20, 
2014,  from 

http  ://www.af  mil/ AboutUs/FactSheets/Displav/tabid/224/Article/104506/f-22- 

raptor.aspx 

Department  of  the  Air  Force.  (2011).  Quick  Reaction  Capabilities  Process.  AFI  63-114. 
Washington;  HQ  USAF 

Department  of  the  Navy.  (2005).  Navy  CVN-2I  Aircraft  Carrier  Program:  Background 
and  Issues  for  Congress.  Retrieved  February  20,  2014,  from 
http://www.historv.navv.mil/librarv/online/navvcvn2 1  .htm 

Department  of  the  Navy.  (2013).  Aircraft  carriers  -  CVN.  Retrieved  February  20,  2014, 
from  http://ipv6.navv.mil/navvdata/fact  display. asp?cid=4200&tid=200&ct=4 

Ferrara,  J.  (1996).  DoD's  5000  documents:  Evolution  and  change  in  defense  acquisition 
policy.  Acquisition  Review  Quarterly,  Fall  1996 


60 


Gansler,  J.  S.,  &  Hughes,  B.  (2009).  Defense  Science  Board  Task  Force,  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  ‘Fulfillment 
of  Urgent  Operational  Needs,  9  July  2009 

Halcomb,  E.  J.,  &  Davidson,  P.  M.  (2006).  Is  Verbatim  Transcription  Of  Interview  Data 
Always  Necessary?  Applied  Nursing  Research,  19(1),  38-42. 

Haskins,  C.,  Forsberg,  K.,  &  Krueger,  M.  (2007).  Systems  Engineering  Handbook. 
INCOSE.  Version  3.1 

Johnson,  K.  M.  (2010).  Tailoring  Systems  Engineering  Processes  For  Rapid  Space 
Acquisitions.  NAVAE  POSTGRADUATE  SCHOOL  MONTEREY  CA 

Joint  Chiefs  of  Staff.  (2012).  Joint  Capability  Integration  And  Development  Systems, 
CJCS3170.01H 

Lepore,  D.,  &  Colombi,  J.  (2012).  Expedited  systems  engineering  for  rapid  capability 
and  urgent  needs.  Final  Report,  SERC  RT-34 

Lund  Research  Ltd.  (2012).  Snowball  Sampling.  Retrieved  1/13,  2014,  from 
http://dissertation.laerd.com/snowball-sampling.php 

Merriam,  S.  B.  (2009).  Qualitative  Research:  A  Guide  to  Design  and  Implementation 
John  Wiley  &  Sons. 

Office  of  the  Undersecretary  of  Defense  for  Acquisition  Technology  &  Logistics.  (2008). 
Operation  of  the  Defense  Acquisition  System.  DoDI  5000.02 

Office  of  the  Undersecretary  of  Defense  for  Acquisition  Technology  &  Logistics.  (2013). 
Operation  of  the  Defense  Acquisition  System,  Draft.  DoDI  5000.02 

Oliver,  D.  G.,  Serovich,  J.  M.,  &  Mason,  T.  L.  (2005).  Constraints  and  opportunities  with 
interview  transcription:  Towards  reflection  in  qualitative  research.  Social  Forces, 
84(2),  1273-1289. 

Pickard,  A.,  &  Nolan,  A.  (2010).  When  Is  Enough  Enough?  Tailoring  Processes  in 

Systems  Engineering.  Paper  presented  at  the  20th  INCOSE  International  Symposium 
in  Chicago, 

Smith,  A.  (2011).  Rapid  development:  A  content  analysis  comparison  of  literature  and 
purposive  sampling  of  AFRL  rapid  reaction  projects.  (AFIT/GRD/ENV/I  ID-0 1)  AIR 
FORCE  INSTITUTE  OF  TECHNOLOGY  WRIGHT-PATTERSON  AFB  OH 
SCHOOL  OF  ENGINEERING  AND  MANAGEMENT. 

Sullivan,  M.  J.  (2009).  Defense  Acquisitions.  Rapid  Acquisition  ofMRAP  Vehicles,  8 
October  2009 


61 


Appendix  A  Interview  Protocol 


Project:  Tailoring  Systems  Engineering  for  Rapid  Acquisition 
Time  of  Interview:  Place: 


Data:  Interviewer:  Interviewee: 


Interview  Procedure 

You  are  being  asked  to  participate  in  a  research  study  investigating  how  the  Air  Force 
completes  rapid  acquisition  and  the  System  Engineering  processes  used.  The  purpose  of 
this  study  is  to  understand  the  current  rapid  acquisition  processes  used  by  the  USAF  and 
how  programs  are  being  tailored  to  meet  expedited  timelines.  This  can  illuminate  any 
areas  of  deficiency  in  the  processes  and  identify  linkages  to  better  manage  rapid 
acquisition.  During  this  interview,  you  will  be  asked  to  respond  to  several  open-ended 
questions.  You  may  choose  not  to  answer  any  or  all  of  the  questions.  The  procedure  will 
involve  taping  the  interview,  and  the  tape  will  be  transcribed  verbatim.  Your  results  will 
be  confidential,  and  you  will  not  be  indentified  individually. 

Questions 

1 .  What  experience  do  you  have  with  rapid  acquisition? 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology  push.)? 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

7.  What  SE  activities  did  your  programs  include? 

8.  How  did  you  decide  which  processes  to  include? 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline  and 
add  activities  or  do  you  start  with  a  standard  'whole'  program  and  remove  activities? 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is  tailored? 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

Closing 

Thank  you  for  participating  in  this  interview.  I  appreciate  you  taking  the  time  to  do  this. 
We  may  contact  you  in  the  future  for  the  purpose  of  follow  up  interviews.  Again,  let  me 
assure  you  of  the  confidentiality  of  your  responses.  If  you  have  any  questions,  please  feel 
free  to  contact  me  by  telephone  at  XXX-XXX-XXXX. 
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Appendix  B  Systems  Engineering  Processes  Handout 


Process 

Level 

SE  Activity 

TP-1 

1 

TP-1  (Stakeholder  Requirements  Development) 

TP-1 

2 

Establish  Communications  with  Stakeholders 

TP-1 

2 

Identify  Project  Constraints 

TP-1 

2 

Determine  Required  Capabilities 

TP-1 

2 

Determine  Desired  Performance 

TP-2 

1 

TP-2  (Requirements  Analysis) 

TP-2 

2 

Analysis  Preparation 

TP-2 

2 

Perform  Functional  Analysis 

TP-2 

2 

Perform  Behavioral  Analysis 

TP-2 

2 

Perform  Environmental  Analysis 

TP-2 

2 

Design  Factors  Analysis 

TP-2 

2 

Develop  Functional  Architecture 

TP-3 

1 

TP-3  (Architecture  Design) 

TP-3 

2 

Define  Design  Problem 

TP-3 

2 

Generate  Alternative  Design  Solutions 

TP-3 

2 

Evaluate  Design  Alternatives 

TP -4 

1 

TP-4  (Implementation) 

TP-4 

2 

Generate  Implementation  Strategy 

TP-4 

2 

Fabricate  Hardware 

TP-4 

2 

Code  Software 

TP-4 

2 

Conduct  Unit  Testing 

TP-4 

2 

Conduct  Training 

TP-4 

2 

Prepare  for  Integration 

TP-5 

1 

TP-5  (Integration) 

TP-5 

2 

Determine  Integration  Process 

TP-5 

2 

Conduct  Assembly  /  Integration  of  System 

TP-5 

2 

Relevant  Environment 

TP-6 

1 

TP-6  (Verification) 

TP-6 

2 

Plan  Verification 

TP-6 

2 

Execute  Verification 

TP-7 

1 

TP-7  (Validation) 

TP-7 

2 

Plan  Validation 

TP-7 

2 

Execute  Validation 

TP-8 

1 

TP-8  (Transition) 

TP-8 

2 

Identify  Transition  Opportunities 

TP-8 

2 

Qualify  Production  Item 

TP-8 

2 

Execute  Transition 

TMP-1 

1 

TMP-1  (Decision  Analysis) 

TMP-1 

2 

Identify  Strategy  for  Making  Decision 

TMP-1 

2 

Execute  Decision  Making  Strategy 

TMP-2 

1 

TMP-2  (Technical  Planning) 

TMP-2 

2 

Plan  Systems  Engineering 

TMP-2 

2 

Implement  Technical  Plan 

TMP-2 

2 

Evaluate  Plan  to  Address  Needs 

TMP-3 

1 

TMP-3  (Technical  Assessment) 
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TMP-3 

2 

Prepare  for  Technical  Assessment 

TMP-3 

2 

Perform  Technical  Assessment 

TMP-4 

1 

TMP-4  (Requirements  Management) 

TMP-4 

2 

Determine  Roies/Responsibilities  During  Reqs  Generation  Process 

TMP-4 

2 

Define  System  Capabilities  and  Performance  Objectives 

TMP-4 

2 

Validate  Requirenients  Development  Process 

TMP-4 

2 

Ensure  Requirements  Feasibility  artd  Validity 

TMP-4 

2 

DcKument  Requirements 

TMP-4 

2 

Ensure  Traceability  of  Requirements 

TMP-4 

2 

Establish  Process  for  Requirements  Changes 

TMP-5 

1 

TMP-5  (Risk  Management) 

TMP-5 

2 

Risk  Planning 

TMP-5 

2 

Risk  Identihcation 

TMP-5 

2 

Risk  Analysis  (Qualitative  &  Quantitative) 

TMP-5 

2 

Risk  Handling 

TMP-5 

2 

Risk  Monitoring 

TMP-5 

2 

Risk  Documentation 

TMP-6 

1 

TMP-6  (Configuration  Managenoent) 

TMP-6 

2 

Develop  Configuration  Baselines 

TMP-6 

2 

Establish  Configuration  Change  Control  Plan  (Establish  configuration  control  cycle 
that  incorporates  evaluation,  approval,  validation,  and  verification  of  change 
requests) 

TMP-6 

2 

Develop  and  Maintain  Configuration  Control  Documentation 

TMP-6 

2 

Maintain  Configuration  Baselines 

TMP-7 

1 

TMP-7  (Technical  Data  Management) 

TMP-7 

2 

Develop  Data  Managenient  Plan 

TMP-7 

2 

Determine  /  Define  System  Relevant  Information 

TMP-7 

2 

Identify  System  Data  to  Purchase 

TMP-7 

2 

Determine  Data  Protection  Requirements 

TMP-7 

2 

Address  Long-term  Data  Storage  Requirements 

TMP-7 

2 

Record  Program  Data 

TMP-7 

2 

Make  Project  Data  Available 

TMP-8 

1 

TMP-8  (Interface  Management) 

TMP-8 

2 

Define  Interface  Requirements  and  Control  Methods 

TMP-8 

2 

Develop  System  Interface  Control  Methods 

TMP-8 

2 

Generate  Interface  Control  Docunientation 

TMP-8 

2 

Utilize  Interface  Controls 
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Appendix  C  List  of  Codes  Used  In  Open  Coding 


Code 

Code 

Acceptable  Solution  Level 

Causality 

Aequisition  Process 

Challenges 

Aequisition  Process  Iterations 

COA  Development 

Aequisition  Tailoring 

Communieation 

Add  as  they  go 

Configuration  Control 

Add  to  baseline 

Configuration  Management 

Ad-hoe  Proeess 

Constraints 

API  Proeess 

Contraeting 

AFRL  Eight  Questions 

Contraetor  Management 

AFRL  Process 

Decision  Analysis 

Arehiteeture  Design 

Difference  between  user  and  aequisition 

Architeeture  Design  wasn't  done 

Doesn't  follow  CPS 

Attribute  Interaetions 

Doesn't  look  at  lifeeyele 

Attributes:  Communieations 

Don't  have  to  proeure  eertifieations 

Attributes:  Configuration 

Don't  respond  to  UONs  or  JUONs 

Attributes:  Cool  faetor 

Emotional  Response  to  SE  keywords 

Attributes:  Design 

Engineering  Reviews 

Attributes:  expeeted  level  of  effort 

Example  of  Communieation  Start 

Attributes:  Manufaeturing  readiness 

Example  of  paperwork  issues 

Attributes:  MDA 

Example  of  proeess 

Attributes:  Money 

Experienee 

Attributes:  personnel 

Expert  driven 

Attributes:  Requirements 

Federated  System 

Attributes:  Risk  Management 

Final  Deeision 

Attributes:  Sehedule 

Focus  on  design 

Attributes:  Scope 

Formal  management 

Attributes:  Small  Seope 

Freeze  Requirements 

Attributes:  Speed  and  Risk 

How  to  decide  tailoring 

Attributes:  Sustainment 

IDIQ  Contraet 

Attributes:  synehronization 

Implementation 

Attributes:  System  Integration 

Increase  of  SE  with  teeh  level 

Attributes:  Teeh  Level 

Incremental  aequisition 

Attributes:  User 

Ineremental  aequisition 

Bottom  Up  Tailoring 

Informal  Proeess 
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Code 

Code 

Can  do  both  top  down  and  bottom  up 

Informal  Proeess 

Informal  Requirements  Definition 

No  Eormal  Risk  Management 

Initiation 

No  JUON  or  UONs 

Integration 

No  QRC 

Interaetions:  Verifieation  and  Validation  to 
Requirements 

No  Requirement  Communieation 

Interfaee  management 

Non  63-1 14 

Interfaee  Management  with  Integration 
Interaetion 

Non  iterative  projeets 

Issues  with  aequisition 

Not  as  mueh  rigor 

Issues  with  Big  Aequisition  Risk 
Management 

Not  Good  at  Requirements  Definition 

Iteration 

Not  iterative 

Iterative  SE 

NVA  in  QRC 

JIEDDO  Requirements 

One  type  of  proeess 

JTCD  experienee 

Operational  Needs 

JUON  Start 

Organizations  with  eontaets  in  plaee 

JUON/UON 

Paperwork  Issues 

Eab  doesn't  go  looking  for  projeets 

part  of  the  proeess  versus  whole  proeess 

Eab  experienee 

PDR / CDR 

Eab  start  with  teeh  push 

People  vs.  Proeess 

Eaek  of  lifeeyele  viewpoint 

Performanee  Management 

Eaek  of  risk  management 

Personal  Communieations  with  leadership 

Eaek  of  SE  bad 

Personal  Experienee 

less  doeumentation 

Personality  driven 

Eess  SE  more  problems 

Poliey 

Eevel  for  transition 

Positive  Output 

Eevel  of  effort 

Priority 

Eow  teeh  readiness  to  bad  outeome 

Priority  Eor  Program 

Management 

Proeess 

Mateh  program  to  existing  requirements 

Proeess  BAAs 

Maturity  and  risk  define  tailoring 

Proeess  Coneems 

minimal  iterations  eompleted 

Proeess  improvement 

Minimal  named  SE 

Proeess  Iteration 

Multiphase 

Produetion  time  frame 

Multiple  Attempts 

Program  Attributes:  Teeh  and 
Manufaeturing  readiness 

Need  for  Tailoring  SE 

Program  Manager  Experienee 

Negative  effeet  of  tailoring 

Program  Start 

No  eorporate  memory 

Projeet  planning 
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Code 

Code 

Project  time  frame 

SE  interactions 

Project  vs.  program 

SE  interactions  with  timelines 

Purchase 

SE  Iteration 

QRC 

SE  not  stressed  on  at  6-1,  2 

Rapid  Acquisition  Definition 

SE  problems 

Rapid  Culture 

SE  Process 

RDIF 

SE  Tailoring 

Reputation  helps  start  programs 

SE  TP  activities 

Requirement 

Set  apart  from  standard  acquisition 

Requirement  Creep 

Short  Usage  Period 

Requirement  Definition  Interaction 

Single  iteration 

Requirement  Growth 

Single  SE  iteration 

Requirements  Analysis  wasn't  done 

Single  time  solution 

Requirements  definition 

Smaller  scale  SE 

Requirements  Development 

Spiral  Process 

Requirements  Generation 

Stakeholder  Communication 

Requirements  Interactions 

Starts  COCOM  Communication 

Requirements  management 

Starts:  Tech  Push 

Requirements  Start 

Starts:  UON/JUON 

Requirements 

Streamline  Contracts 

Reviews  Interactions 

Streamline  vs.  Eliminate 

Rigor 

Streamlined  paperwork:  ADM  solution 

Risk  for  tailoring 

Success  criteria 

Risk  interaction 

Support 

Risk  Management 

Synergy  in  workforce 

Risk  Management  Interaction 

System  Design 

Risk  Management  to  Overall  Project 

Success  Interaction 

Systems  Engineering  Process  Selection 

SAP  initiates  based  on  JUON 

Tailor  based  on  money 

Savings 

Tailor  based  on  priority 

SBIR  Process 

Tailor  buying  strategy 

Schedule  and  risk  decision 

Tailor  contract  due  to  money 

Schedule  risk 

Tailor  though  out  process 

SE 

tailored  based  on  level  of  effort 

SE  concepts 

Tailored  based  on  Maturity 

SE  guidance 

Tailoring 

Tailoring  Contractor 

Time  based  iteration 

Tailoring  Effects 

Timeline 
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Code 

Code 

Tailoring  risk 

Timelines  and  effort  level  determine 
tailoring 

Tech  Assessment 

TMP  based  on  size 

Tech  Data 

TMP  Processes 

Tech  data  and  contracting 

Tools  for  acquisition 

Tech  Data  Management 

Top  cover 

Tech  data  usage 

Top  Down 

Tech  Integration 

Top  Down  Focus 

Tech  planning  interaction 

Top  down  tailoring 

Tech  planning  process  for  acquisition 
projects 

Transition 

Tech  Pull 

Transition 

Tech  Push  transition 

Transition  of  Program  to  Program  of 

Record 

Technical  Assessment 

Tried  to  extrapolate  too  far 

Technical  Data  Management 

TRL  interacts  with  program  success 

Technical  Maturity 

UON  process  for  RDIF 

Technical  Planning 

UON/JUON 

Technical  Readiness 

Use  design  discipline  in  decision  making 

Technology  maturity 

Use  Existing  JUONS 

Testing 

Verification  /  Validation 

The  process  they  use  is  tailored 

View  on  SE 

Thoughts  on  APIs 

View  on  Traditional  Acquisition 

Time  /  Schedule  decision 

When  to  Stop  Tailoring 
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Appendix  D  Interview  Transcripts 
Summary  of  Interview  #1 

1 .  What  experience  do  you  have  with  rapid  acquisition? 

Extensive  experience  in  acquiring  technology  solutions  rapidly  in  the 
laboratory  environment.  Minimal  experience  with  traditional  acquisition 
programs. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

Core  process  3  in  the  laboratory. 

3.  Does  your  office  follow  the  QRC  process  defined  in  API  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Informal  requirement  generation  from  leadership  meeting  with  the  warfighter. 
UON  and  JUON  used.  Both  technology  pull  and  technology  push. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

Somewhat  iterative,  depending  on  the  program  requirements. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

It  can  be  either  incremental  or  a  single  time  solution.  Some  are  small 
programs  with  limited  usage;  others  are  multiple  iterations  with  each  giving 
some  new  capability. 

7.  What  SE  activities  did  your  programs  include? 

Preliminary  design  reviews,  critical  design  reviews,  configuration  control. 
They  complete  risk  management  through  safety  review  boards. 

8.  How  did  you  decide  which  processes  to  include? 

They  have  guidance  in  the  APRLIs. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

When  everyone  is  satisfied  they  stop  the  SE  activities.  They  have 
independent  reviews  for  programs  at  different  points  to  determine  if  they  are 
ready  to  progress  to  other  phases.  They  also  weigh  the  return  on  investment, 
small  programs  with  little  ROI  are  much  less  rigorous. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

A  lack  of  requirements  analysis  impacts  the  overall  program  and  meeting  the 
user’s  needs.  Has  seen  “synergistic  flow  of  processes  from  the  engineering 
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review  level  up  through  teehnical  and  safety  risk  assessments  and  program 
reviews.” 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

His  projects  have  used  support  contractors  that  have  the  ability  to  do  things 
quickly  without  the  bureaucracy  associated  with  government  of  military 
employees. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

Based  upon  the  scope  of  the  program.  The  larger  the  program  the  more 
different  activities  and  processes  must  be  completed  ad  to  higher  levels  of 
rigor.  Also  based  upon  the  experience  of  the  PM  and  project  team. 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

99%  of  the  time  it’s  a  positive  effect  on  the  program  with  respect  to  time  and 
schedule  constraints. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

Depends  on  the  PM  and  their  experience  level. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Money,  time  frame,  who  the  stakeholders  are,  whose  money  it  is  and  what  the 
contract  says,  technology  readiness,  manufacturing  readiness. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

Technology  readiness  to  the  acceptable  solution  level.  They  are  happy  if  they 
can  get  to  a  90%  solution  level,  sometimes  the  customer  accepts  as  low  as 
50%  success  as  they  are  currently  not  able  to  get  any  success  or  relief  from 
their  capability  gap.  Requirements  definition  to  the  overall  success  of  the 
program. 
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Summary  of  Interview  #2 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Multiple  rapid  technology  development  programs  in  support  of  the  operational 
warfighter. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

Used  an  ad-hoc  process.  Considers  the  process  to  be  people  oriented  versus 
process  oriented.  “If  everyone  could  be  special  operations,  then  we  wouldn’t 
need  the  infantry.” 

3.  Does  your  office  follow  the  QRC  process  defined  in  API  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Sometimes  it’s  started  by  an  UON/JUON  other  times  it’s  started  by  informal 
requirements  generation  and  phone  calls  between  people  that  know  each 
other. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

Very  iterative.  Many  field  testing’s  with  the  end  users  before  final  products 
are  produced. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

Both  incremental  and  single  time  solutions  used  based  upon  the  program 
requirements. 

7.  What  SE  activities  did  your  programs  include? 

Doesn’t  complete  SE  activities  by  name.  Views  design  process  as  a  loop; 
design,  build,  test. 

8.  How  did  you  decide  which  processes  to  include? 

Inclusion  of  activities  is  based  upon  experience  of  PM  and  SMEs  of  the 
technology  being  used  in  the  programs. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

N/A  due  to  previous  answers. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

N/A  due  to  previous  answers. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 
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Removed  programs  from  standard  laboratory  proeesses.  As  long  as  they  were 
designated  a  Core  Proeess  3  program,  they  were  able  to  waive  many  of  the 
proeess  requirements  levied  by  their  organizations  on  standard  programs. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

Based  upon  PM  experienee  and  judgment. 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

Positive  effect  on  schedule. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add 

They  start  with  a  minimal  program  and  add  activities  until  the  PM,  SMEs,  and 
customer  is  comfortable  with  the  program. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

“A  need  that  both  the  engineers  and  warfighters  agree  is  achievable.”  Strong 
communications. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

“So  if  you  are  going  to  do  a  rapid  reaction  project,  the  technology  itself  has 
been  mature.” 
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Summary  of  Interview  #3 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

PM  for  rapid  program  on  major  USAF  air  frame  in  a  traditional  program 
office.  Also  completed  an  USAF  exercise  for  a  QRC  which  included  all  the 
coordination  issues  of  a  rapid  acquisition  program  but  did  not  execute  the 
design  portion. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

The  QRC  process  in  the  exercise  and  a  QRC-like  process  for  the  rapid 
acquisition  program. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Exercise  was  a  UON  and  the  actual  rapid  acquisition  program  was  started  by 
the  CSAF  designating  the  need  for  the  program. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

The  process  is  iterative  in  that  they  deliver  a  certain  set  of  capability  with  the 
first  version  and  then  modify  that  to  increase  the  capability  in  future  iterations. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

It’s  an  iterative  process. 

7.  What  SE  activities  did  your  programs  include? 

“Well  the  one  way  to  do  it  in  a  disciplined  manner  is  to  follow  a  systems 
engineering  process  that  is  tried  and  true  and  the  [office]  has  one  I  just  wasn’t 
aware  of  it.”  They  worked  the  design  technical  process,  stakeholder 
requirements  definition,  and  technical  reviews. 

8.  How  did  you  decide  which  processes  to  include? 

Through  communication  with  the  contractor  and  based  upon  the  experience  of 
the  PM. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

“I  say  as  we  moved  closer  to  each  one  of  those  phases  we  took  a  deeper  look 
at  them  to  make  sure  are  we  doing,  or  are  we  meeting  the  minimum  standards 
of  what  we  need  to  do  in  each  one  of  these  phases.  So  we  did  kind  of  go  back 
and  double  check  to  make  sure  that  we  were  doing  the  right  amount  of  due 
diligence  in  each  one  of  those  phases  of  the  project.” 
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10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

The  lack  of  rigor  in  certain  activities  caused  the  program  to  have  to  repeat 
them.  Later  in  the  program,  the  rigor  was  stressed  and  the  reviews  and  testing 
went  much  better. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

By  reviewing  how  they  were  going  to  be  reviewing  the  program  they  were 
able  to  tailor  how  certain  products  were  being  reported  and  ensured  that  the 
program  met  the  key  requirements. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

It’s  based  upon  the  experience  of  the  PM  and  program  staff 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

It  expedited  the  schedule  and  allowed  the  program  to  meet  the  needs  stated  by 
the  CSAF. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

The  respondent  reported  that  it  was  both.  They  knew  the  main  processes  that 
their  organization  would  use  on  a  traditional  program  and  went  down  from 
there,  while  they  also  went  from  a  detailed  schedule  and  built  up  from  that. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Cost,  test  schedule,  program  priority,  and  system  integration. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

Design  requirements  were  defined  in  such  a  way  to  minimize  the  integration 
necessary  with  the  aircraft  systems  as  a  whole. 
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Summary  of  Interview  #4 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Contractor  supporting  AFRL.  Has  conducted  multiple  programs  in  support  of 
the  rapid  acquisition  cell  at  AFRL  in  support  of  urgent  needs. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

The  process  is  more  of  an  ad-hoc  process.  Considers  the  process  to  be  a 
streamlined  version  of  AFRL  Core  Process  3. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Most  start  by  receiving  a  UON/JUON.  However  some  programs  are 
technology  push  coming  out  of  the  lab. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

Not  all  that  iterative. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

The  respondent  considers  most  programs  to  single  time  solutions.  Each 
program  might  build  on  previous  ones,  but  not  in  a  planned  incremental 
solution  set. 

7.  What  SE  activities  did  your  programs  include? 

Requirements  Definition,  interface  management,  assessments  on  capabilities 
required,  and  risk  management. 

8.  How  did  you  decide  which  processes  to  include? 

Trades  are  made  between  activities  based  upon  the  experience  of  the  PM  and 
team  along  with  the  schedule  of  the  associated  program. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

Somewhat  iterative. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

The  lack  of  configuration  control  cause  issues  when  the  user  came  back  on  a 
program  and  asked  for  more  of  the  items.  Eack  of  configuration  control 
meant  that  the  items  they  received  were  not  fully  compatible  with  the  original 
set  of  equipment. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 
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Tailoring  was  completed  based  upon  the  experienee  of  the  PM.  Tailoring  was 
eondueted  based  upon  what  the  programs  require,  then  understanding  what 
tools  had  to  be  brought  to  bear  on  the  problem.  This  would  dietate  what  had 
to  be  eompleted  based  upon  the  time  allowed. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

“When  it  starts  hindering  the  exeeution  you’ve  gone  too  far.” 

13.  What  effeets  did  tailoring  have  on  the  overall  projeet? 

Normally  a  positive  effeet  on  reaehing  the  sehedule  eonstraint. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  aetivities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
aetivities? 

It  ean  be  both  a  top  down  methodology  or  a  bottom  up  depending  on  the 
program.  Due  to  the  ambiguous  nature  of  some  of  the  problems,  they  will  add 
proeess  as  they  go. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Teehnology  readiness,  integration,  money,  and  time/sehedule. 

16.  What  interaetions  are  observed  between  the  attributes  and  the  outeome  of  the 
program? 

The  respondent  saw  non-linear  relationships  between  requirements  to  eost  and 
other  implieations. 
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Summary  of  Interview  #5 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Came  from  the  test  community  to  the  laboratory.  In  the  respondents 
leadership  position,  they  review  the  programs  that  are  being  conducted  under 
his  purview. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

The  lab  SE  process  based  upon  AFRLIs  and  the  eight  SE  questions  discussed 
there. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AEI  63-114? 

The  respondent  does  not  work  QRC  processes.  Some  programs  might  be  part 
of  a  larger  subset  of  activities  that  fall  under  a  QRC  process,  but  there  internal 
process  is  not  regulated  by  the  QRC  process. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Technology  push  and  the  needs  of  the  warfighter.  Their  discussions  with  the 
warfighter  might  cause  a  JUON  or  UON  to  be  created  to  acquire  the 
technology  that  they  have  been  working  on. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

The  processes  iteration  depends  on  the  programs  being  conducted.  Some  are 
single  iterations  while  others  are  multiple  iterations. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

Both.  Some  programs  are  technology  demonstrations  that  once  they  work  the 
laboratory  is  finished  with  the  program,  while  others  are  incremental  upgrades 
to  programs  already  fielded. 

7.  What  SE  activities  did  your  programs  include? 

The  lab  is  more  focused  on  technical  processes  than  technical  management 
processes.  Verification  and  validation  are  a  “big  part  of  what  we  do  here.” 
Configuration  management  is  also  conducted. 

8.  How  did  you  decide  which  processes  to  include? 

The  front  office  has  begun  to  require  more  systems  engineering  to  increase  the 
rigor  in  many  projects. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

Iterations  depends  on  the  approval  authority.  If  the  approval  authority  is  ok 
with  the  work  completed  then  they  will  allow  the  program  to  move  onto  the 
next  phase,  where  as  if  they  are  unhappy  with  the  level  the  program  is  at  they 
will  require  them  to  go  back  and  conduct  more  work  to  correct  any  issues 
discovered  during  the  reviews. 
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10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

There  is  an  interaction  between  Requirements  Definition  and  Verification  and 
Validation.  Also  Risk  Management  interacts  with  the  program,  along  with 
interface  management. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

More  of  streamlining  steps  then  eliminating  them. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

Level  of  effort  drives  some  tailoring  aspects.  If  the  program  is  small  you  will 
not  need  or  be  able  to  conduct  as  many  activities  as  on  larger  programs.  Also, 
what  does  the  risk  management  strategy  state  is  the  high  risk  areas? 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

Generally  positive  effects  for  tailoring  programs,  when  conducted  correctly. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

Somewhere  between  the  top-down  and  bottom  up  methodologies.  They  have 
an  idea  of  what  needs  to  be  done  and  then  build  their  processes  from  their 
based  upon  the  program. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Technology  maturity,  budget,  and  system  interactions. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

Risk  impacts  many  of  the  program  decisions. 
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Summary  of  Interview  #6 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Multiple  rapid  projects  under  AFRL  and  JIEDDO. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

The  acquisition  process  used  was  an  ad-hoc  process.  Didn’t  really  follow 
AFRLIs  or  AFI63-114. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Informal  requirements  definition  by  having  the  respondent  try  new 
technologies  that  might  work  for  a  general  problem  set.  Afterwards,  the 
respondent  discussed  what  he  found  with  the  warfighter  to  see  if  it  would 
work  and  should  be  fielded. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

It  is  an  iterative  solution  process. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

The  respondent  stated  that  they  never  walk  away,  and  some  programs  are 
iterative  upgrades  to  older  programs. 

7.  What  SE  activities  did  your  programs  include? 

Skipped  due  to  interviewee  not  receiving  the  SE  process  list  before  interview. 

8.  How  did  you  decide  which  processes  to  include? 

No  answer  provided. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

No  answer  provided. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

No  answer  provided. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

Programs  are  managed  informally  and  the  respondent  has  had  to  inject  more 
rigor  into  the  processes  used. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

Based  upon  the  experience  of  the  PM  and  program  team. 

13.  What  effects  did  tailoring  have  on  the  overall  project? 
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Couldn’t  answer  directly.  Stated  that  his  organization  relies  on  the  “contractor 
to  really  carry  the  ball  on  the  program  management  and  systems  engineering 
oversight.” 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

Reported  using  a  bottom-up  methodology. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

The  respondent  didn’t  really  evaluate  programs  based  upon  discriminates,  the 
only  one  that  they  discussed  was  money. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

Having  the  contract  already  in  place  so  that  you  can  add  the  activities  for  the 
program  under  an  existing  contract  expedites  the  process. 
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Summary  of  Interview  #7 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Early  career  was  spent  in  the  lab  working  rapid  acquisition  programs.  Current 
job  is  at  a  traditional  program  office  working  both  traditional  and  rapid 
acquisition  programs  for  an  airframe. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

Reported  using  an  ad-hoc  process. 

3.  Does  your  office  follow  the  QRC  process  defined  in  API  63-114? 

Worked  a  two  or  three  QRC  programs  when  the  API  was  first  published  but 
now  most  programs  are  worked  as  ad-hoc  processes. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

At  APRP  it  was  predominantly  technology  push.  At  the  SPO  it  is  driven  by 
UON/JUON  and  CSAP  directed  programs 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

Not  an  iterative  process.  If  you  have  to  iterate  you’ve  done  something  wrong 
that  is  causing  rework. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

Rapid  acquisition  can  be  both.  Respondent  stated  that  they  would  prefer  to 
push  out  multiple  small  iterations,  but  some  programs  require  larger,  more 
complex  answers  that  reduce  the  amount  of  iterations  possible. 

7.  What  SE  activities  did  your  programs  include? 

Reported  using  almost  all  the  technical  processes,  some  of  the  technical 
management  processes. 

8.  How  did  you  decide  which  processes  to  include? 

Inclusion  is  based  upon  the  schedule  and  properties  of  the  program. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

A  perfect  run  through  the  processes  is  expected. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

If  the  requirements  are  not  flushed  out  in  the  beginning  then  you  lose  the 
opportunity  to  create  some  capabilities  on  the  back  end  of  the  program  due  to 
the  inability  to  go  back  on  some  programs. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

If  the  process  is  not  required  by  federal  statue,  then  the  organization  does  not 
complete  it.  The  use  of  undefmitized  contracts  was  approved. 
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12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

It  is  usually  “what  ean  1  get  done  in  the  time  1  have.” 

13.  What  effeets  did  tailoring  have  on  the  overall  project? 

Any  standard  process  that  isn’t  completed  increases  the  risk  of  the  program. 

But  the  experience  level  of  the  PM  and  team  help  to  attenuate  most  issues. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

They  start  with  a  hare-bones  baseline  and  then  add  processes  as  they  progress 

through  the  planning  and  execution  of  the  program. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Speed  and  risk. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

The  speed  of  the  programs  dictates  that  they  rarely  do  lessons  learned. 

Adding  in  the  turnover  of  personnel  and  they  reported  that  they  make  the 

same  mistakes  over  and  over. 
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Summary  of  Interview  #8 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Conducted  multiple  rapid  acquisition  programs  at  AFRL. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

In  the  respondents  previous  experience,  they  reported  conducting  QRC 
processes  along  with  rapid  programs  that  fell  under  the  purview  of  the  AFRL 
process. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

None  of  the  laboratory  rapid  acquisition  programs  are  designated  as  a  QRC 
programs. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

UON/JUON  initiations  along  with  technology  push  coming  from  the 
laboratory. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

It  is  iterative. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

It’s  normally  considered  an  incremental  process. 

7.  What  SE  activities  did  your  programs  include? 

Requirements  definition,  implementation,  integration,  verification  and 
validation,  technical  assessment,  and  interface  management. 

8.  How  did  you  decide  which  processes  to  include? 

Inclusion  was  based  upon  the  experience  of  the  PM  and  program  team. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

The  processes  are  iterative,  especially  if  you  fail  part  of  the  verification  or 
validation  portion  of  the  program. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

Over  extrapolation  of  verification  in  one  environment  to  another  environment. 
Another  reported  interaction  was  risk  management  to  overall  success  of  the 
program. 

1 1 .  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

Tailoring  was  based  upon  schedule,  the  analysis  of  the  problem  and  the 
maturity  of  the  technology  being  used  in  the  program. 
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12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

[Inadvertently  skipped  by  interviewer.] 

13.  What  effeets  did  tailoring  have  on  the  overall  project? 

It’s  normally  positive,  but  can  have  a  negative  effect  if  you  end  up  tailoring 

out  an  activity  you  realize  you  needed  later  in  the  program. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

It’s  a  bottoms  up  methodology. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Technology  readiness,  manufacturing  readiness. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

“If  you  haven’t  done  the  requirements  analysis  part  it’s  not  going  to  turn  out 

so  hot.  Your  stuff  isn’t  going  to  turn  out  to  have  the  performance  you  need. 

And  risk  management  is  pretty  much  the  same.” 
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Summary  of  Interview  #9 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Both  respondents  had  experience  in  rapid  acquisition.  Respondent  1  worked  in  the 
civilian  world  conducting  rapid  development  and  prototype  before  coming  to 
government  service  and  working  in  what  should  be  considered  a  rapid 
development  program  office.  Respondent  2  was  in  private  industry  working  and 
upon  starting  to  work  as  a  government  civilian  had  conducted  multiple  rapid 
acquisition  projects. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 
Could  be  considered  an  ad-hoc  process  or  an  organizational  defined  best  practices 
based  upon  experience  of  project  teams  and  the  PM. 

3.  Does  your  office  follow  the  QRC  process  defined  in  API  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Informal  requirements  generation  with  normally  a  phone  call  or  email  from 
another  program  who  requires  their  help  in  the  design  process  and  problem 
solving. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

The  process  is  iterative  in  that  if  the  original  design  does  not  work  they  are  ok 
with  going  back  and  redesigning  it. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 
Both  respondents  stated  that  they  do  not  do  repetitive  production,  but  they  will  do 
incremental  increases  in  capabilities  if  required. 

7.  What  SE  activities  did  your  programs  include? 

The  respondents  stated  that  while  the  do  SE  like  activities,  they  do  not  do  the 
standard  SE  activities.  Elpon  further  discussion  they  stated  that  they  do 
requirements  analysis,  implementation,  integration,  verification  and  validation, 
technical  reviews,  but  all  activities  have  different  names  than  the  standard  SE 
names. 

8.  How  did  you  decide  which  processes  to  include? 

Based  upon  the  PM  and  program  member’s  experiences. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

The  processes  are  iterative  if  required,  but  they  try  for  minimal  rework  to  allow 
expedited  timelines. 
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10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

Their  process  is  flexible,  they  only  remove  processes  when  they  are  not  adding 
value  to  the  programs. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 
They  tailor  their  projects  “with  respect  to  magnitude  or  if  [they]  are  going  to 
outsource  the  work.”  . 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

[Not  clearly  answered  during  interview.] 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

Tailoring  the  process  accelerates  the  schedule. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

When  designing  their  process,  they  ‘threw  out’  the  traditional  acquisition  process 
and  recreated  what  they  felt  was  necessary  based  upon  their  experience. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

They  choose  some  programs  based  upon  the  interest  of  the  organization  with  the 
problem  or  request.  Other  attributes  include  money,  functional  knowledge  area 
required,  if  they  have  the  ability  or  can  contract  someone  who  has  the  ability  to  do 
the  work. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

They  try  to  level  the  work  flow  of  multiple  programs  being  run  concurrently. 
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Summary  of  Interview  #10 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Conducts  rapid  acquisition  programs  in  the  laboratory. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 
Broad  agency  announcement,  AFRLI  63-104  process,  Small  Business  innovative 
research. 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

No. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Mostly  technology  push. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

The  process  is  iterative,  with  each  program  building  on  the  last. 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 
The  planning  process  is  an  annual  event,  so  most  programs  are  iterative  based 
upon  that  annual  review. 

7.  What  SE  activities  did  your  programs  include? 

Do  most  of  the  technical  processes  and  some  technical  management  processes 
including  configuration  control  and  interface  management. 

8.  How  did  you  decide  which  processes  to  include? 

Inclusion  or  exclusion  of  processes  is  based  upon  the  level  of  the  program  in  the 
laboratory  hierarchy  processes  and  the  finish. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

The  SE  processes  are  iterative  and  have  feedback  loops  built  into  them. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 

In  general  the  respondent  stated  that  the  less  SE  rigor  the  more  problems 
programs  have. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 
Tailoring  was  done  by  using  the  BAA  or  SBIR  instead  of  normal  laboratory 
process.  They  also  tailor  based  upon  the  funding  levels  and  if  they  require  to 
incorporate  more  partners  to  increase  funding. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

The  level  of  tailoring  is  based  upon  funding,  time  and  contracting  support. 

13.  What  effects  did  tailoring  have  on  the  overall  project? 
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Tailoring  can  have  a  negative  effeet  in  that  you  will  not  foeus  on  eertain  areas  of 
teehnology.  It  normally  has  a  positive  effeet  in  that  you  ean  reaeh  sehedule 
requirements. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  aetivities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
aetivities? 

They  use  a  bottom  up  methodology  and  have  a  baseline  that  they  use  and  then  add 
to  it  as  required. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

Money,  program  level,  teehnology  level. 

16.  What  interaetions  are  observed  between  the  attributes  and  the  outeome  of  the 
program? 

Teehnology  level  and  program  level  have  the  biggest  impaet  on  overall  success  of 
the  program. 


Summary  of  Interview  #11 


1 .  What  experience  do  you  have  with  rapid  acquisition? 

Worked  in  the  Northrop-Grumman  EF-1 1 1  Systems  Improvement  office  in 
1994,  a  $1B  program.  The  program  was  later  killed  off  by  Congress.  Worked 
in  the  ACC  weapon  system  program  of  record  designated  by  the  CSAF.  This 
is  where  the  interviewee  heard  the  phrase  “when  skating  on  thin  ice  your  best 
asset  is  speed.”  Interviewee’s  office  was  in  charge  of  integrating  the  weapon 
systems. 

2.  What  process  have  you  seen  being  used  to  complete  rapid  acquisition  programs? 

Programs  are  run  ad-hoc,  and  managed  mostly  by  “sheer  will  of  personality” 

3.  Does  your  office  follow  the  QRC  process  defined  in  AFI  63-114? 

No,  the  in-fighting  between  stakeholders  slows  it  down  too  much. 

4.  How  do  these  programs  begin  (i.e.  initiation  by  UON/JUON,  technology 
push...)? 

Customers  come  in  with  a  JUON  in  hand  and  interviewee’s  organization  is 
tasked  by  SAF/AQ. 

5.  How  iterative  is  the  rapid  acquisition  process  that  your  office  uses? 

Interviewee’s  organization  works  for  the  80%  solution  and  worries  about  the 
20%  after  fielding.  This  allows  the  programs  to  cost  less  than  the  100% 
solution  and  to  be  more  agile  to  the  user.  “Perfect  is  the  enemy  of  good 
enough.” 

6.  Do  you  view  rapid  acquisition  as  an  incremental  process  or  a  single  time  solution? 

Incremental  solutions  with  interviewee’s  organization  working  the  systems 
from  cradle  to  grave.  They  are  constantly  chasing  improvements  to  the 
systems. 

7.  What  SE  activities  did  your  programs  include? 

The  DAG  SE  activities  are  not  done  by  name,  but  the  processes  that  they  use 
meet  the  same  needs. 

8.  How  did  you  decide  which  processes  to  include? 

It’s  personality  based  or  expert  driven  instead  of  lack  of  experience  making 
them  beholden  to  a  process.  Requirements  are  expert  driven  versus  process 
driven. 

9.  How  iterative  are  the  SE  activities  used  in  your  programs? 

They  are  all  iterative,  but  they  try  to  minimize  the  iterations  to  accelerate  the 
programs. 

10.  What  interactions  did  you  see  between  the  SE  process  included  or  excluded? 
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Reviews  such  as  PDR  and  CDR  are  done,  but  they  are  a  lot  closer  than  normal 
acquisition,  2-3  months  apart.  80%  solution  at  PDR  with  a  final  solution  at 
CDR.  There  is  causality  in  everything.  Sometimes  the  rigor  is  bad  due  to  the 
situation  of  the  program. 

11.  How  have  projects  you  have  been  involved  in  tailored  the  acquisition  process? 

The  MDA  steps  in  and  tells  them  how  they  will  be  done  or  the  reporting 
requirements. 

12.  How  do  you  determine  to  what  level  a  program  needs  to  be  tailored? 

Add  until  the  customer  quits  asking  about  a  certain  area.  It  depends  on  the 
user  and  the  MDA. 

13.  What  effects  did  tailoring  have  on  the  overall  project? 

Normally  accelerates  the  programs. 

14.  When  determining  how  to  tailor  a  program,  do  you  start  at  a  minimum  baseline 
and  add  activities  or  do  you  start  with  a  standard  ‘whole’  program  and  remove 
activities? 

Interviewee’s  organization  has  a  minimum  baseline  and  then  adds  to  it. 

15.  What  attributes  does  your  organization  use  to  determine  how  a  program  is 
tailored? 

It’s  the  integration  of  known  technology  and  airframe.  Dollar  amounts  and 
who  the  MDA  is.  User,  personnel,  TRL. 

16.  What  interactions  are  observed  between  the  attributes  and  the  outcome  of  the 
program? 

High  tech  readiness  increases  likelihood  of  program  success. 
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